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Five  estimation  methods,  complete  data  only  (CD) ,  mean 
replacement  (MR) ,  complete  pair  variables  only  (PP) , 
regression  (RG) ,  and  expectation-maximization  (EM) ,  were 
compared  with  respect  to  their  accuracy  in  the  estimation  of 
missing  values,  variance-covariance  matrices,  and  regression 
coefficients.  The  effect  on  estimation  accuracy  was 
investigated  under  amounts  of  missing  data  from  both 
independent  and  dependent  variables,  degree  of  data 
skewness,  type  of  missing  data,  average  correlation  of 
variables,  residual  variance,  number  of  independent 
variables,  and  sample  size. 

Overall,  the  EM  is  superior  to  the  rest  of  the 
estimation  methods.  However,  the  MR  method  is  a  very  good 
alternative,  and  in  most  situations  it  performs  just  as  well 
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as  the  EM.  The  CD  method  perforins  as  well  as  the  EM  method 
when  regression  coefficients  are  estimated.  The  performance 
of  all  methods  declines  when  data  are  censored  and  the 
percentage  of  missing  data  is  high  (15%  missing  data) .  The 
PP  method  performs  as  well  as  the  EM  and  MR  methods  when  the 
independent  variables  are  only  slightly  correlated  (average 
correlation  of  0.2) .  The  RG  method  performs  well  when  the 
number  of  independent  variables  is  low  (three  independent 
variables) ;  otherwise  it  is  the  worst  compared  to  the  rest. 
Estimation  is  better  when  data  are  missing  at  random  than 
when  data  are  censored. 
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CHAPTER  1 
INTRODUCTION 


In  the  absence  of  missing  values,  many  multivariate 
statistical  analyses,   including  least  squares  regression, 
factor  analysis,  and  discriminant  analysis,  are  based  on  an 
initial  reduction  of  the  data  to  the  sample  mean  vector,  /i, 
and  sample  variance-covariance  (V-C)  matrix,  t,  of  the 
variables.  When  vector-valued  observations  are  made,  it  is 
not  unusual  for  one  or  more  of  the  elements  in  one  or  more 
of  the  vector-valued  observations  to  be  missing.  The 
question  of  how  to  estimate  S,  n,  and  other  population 
parameters  when  there  are  missing  data  is,  therefore,  an 
important  one.  If  the  number  of  missing  entries  in  the  data 
matrix  is  very  small,  it  may  be  reasonable  to  discard  all 
the  measurements  for  a  given  individual  with  missing  entries 
and  proceed  with  the  analysis  as  if  that  data  had  never  been 
collected.  However,   even  if  the  percentage  of  missing 
observations  is  small,  the  amount  of  data  excluded  by  this 
procedure  can  become  substantial  if  the  number  of  variables 
is  large. 

There  are  three  classes  of  methods  for  handling 
incomplete  data,  namely  the  complete  data  (CD)  method  (Timm, 
1970),  the  indirect  methods,  and  the  direct  methods.  In  the 


CD  method  the  complete  observations  are  used  in  standard 
procedures  such  as  maximum  likelihood  (ML)  or  least  squares 
(LS)  to  estimate  parameters.  In  the  indirect  methods  the 
missing  entries  are  first  estimated  and  then  the  estimated 
and  observed  data  are  used  in  standard  procedures  to 
estimate  the  parameters.  Among  the  indirect  methods  are  the 
mean  replacement  (MR)  method  (Wilks,  1932),  the  complete 
pair  only  (PP)  method  (Glasser,  1964) ,  the  principal 
component  (PC)  method  (Gleason  &  Staelin,   1973) ,  and  the 
regression  replacement  (RG)  method  (Buck,   1960) .  In  the 
direct  methods  both  the  complete  and  incomplete  observations 
are  used  to  estimate  the  parameters  by  a  maximum  likelihood 
procedure. 

Most  early  work  on  direct  estimation  focused  on  certain 
special  patterns  of  missing  observations;  for  example 
Anderson  (1957),  Afifi  and  Elashoff  (1966),  and  Hocking  and 
Smith  (1968)  .   Subsequent  work  by  Orchard  and  Woodbury  (1972) 
did  not  rely  on  special  patterns.  Orchard  and  Woodbury  used 
a  special  case  of  the  Expectation-Maximization  (EM) 
algorithm.  A  variant  on  the  direct  method,  called  pairwise 
maximum  likelihood  (PML) ,   is  to  apply  the  EM  method  to  each 
pair  of  variables  in  order  to  obtain  estimates  of  the 
covariances.  Each  variance  is  typically  estimated  by 
applying  the  usual  variance  estimator  to  available  data  on 
that  variable.  Research  has  been  conducted  (a)  to  compare 
the  CD  method  and  various  kinds  of  indirect  methods  for 
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estimating  Z  and  the  correlation  matrix  (Gleason  &  Staelin, 
1975;  Timm,   1970);    (b)   to  compare  the  CD,  direct,  and 
indirect  methods  for  estimating  polychoric  correlation 
matrices  (Lee  &  Chiu,   1990) ;   (c)   in  which  V-C  matrices  were 
estimated  using  the  CD,  direct  and  indirect  methods  and 
subsequently  either  regression  parameters  (Little,   1988)  or 
factor  loadings  (Brown,   1983;  Finkbeiner,   1979)  were 
estimated;  and  (d)   in  which  both  direct  and  indirect  methods 
have  been  used  to  estimate  the  missing  values  and  then,  with 
the  CD  method  also  included,  regression  parameters  estimated 
(Azen,  van  Guilder,   &  Hill,  1989).  The  methods  were  then 
compared  in  terms  of  their  ability  to  (a)  reconstruct  the  V- 
C  matrix,    (b)   estimate  the  factor  loadings,  or  (c)  estimate 
the  regression  parameters. 

The  Problem 

Methods  for  estimating  E  when  some  data  are  missing  can 
be  divided  into  three  groups,  the  complete  data  method,  the 
direct  methods,  and  the  indirect  methods.   In  the  complete 
data  method,  the  complete  observations  are  used  to  estimate 
Z  by  least  squares  procedure.  The  methods  in  the  second 
group  incorporate  an  assumption  that  the  observed  data  are 
drawn  from  a  multivariate  distribution  of  known  form 
(usually  the  multivariate  normal)   but  with  unknown 
parameters.  The  parameters  are  estimated  using  the  available 
data  through  an  estimation  procedure  such  as  ML.  Although 
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the  logic  of  the  ML  procedure  is  straightforward  and  the 
derived  estimates  have  good  properties  such  as  consistency, 
sufficiency,  efficiency,  and  asymptotic  normality  (Mood, 
Graybill,   &  Boes,   1974),  the  resulting  computational 
procedures  are  generally  complicated  and  time  consuming.  The 
complexity  of  the  computation  is  often  compounded  by  the 
presence  of  the  missing  values  (Anderson,   1957;  Hocking  & 
Smith,   1968) .  The  methods  in  the  third  group  have  no  data 
distributional  assumptions.  The  methods  utilize  the 
information  in  the  observed  values  to  construct  the  missing 
entries.  Once  the  missing  entries  have  been  estimated,  S 
or/and  regression  parameters  are  calculated  through  the  LS 
procedure. 

The  problem  in  this  study  is  threefold: 

1.  To  compare  the  accuracy  of  estimation  when  the 
complete  data  only,  direct,  and  indirect  estimation  methods 
are  used  in  estimating  missing  values. 

2 .  To  compare  the  accuracy  of  estimation  when  V-C 
matrices  are  estimated  by  using  the  complete  data  only, 
direct,  and  indirect  estimation  methods. 

3.  To  compare  the  accuracy  of  estimation  when  estimates 
of  multiple  linear  regression  parameters  are  calculated  by 
LS  after  estimating  the  V-C  matrix  by  the  complete  data 
only,  direct  and  indirect  methods. 

Accuracy  measures  are  presented  in  chapter  3 . 
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Purpose  of  the  Study 

Among  the  direct  V-C  estimation  methods,  the  study  will 
investigate  the  direct  ML  through  the  Expectation- 
Maximization  (EM)  algorithm.  The  indirect  methods  to  be 
studied  are  the  mean  replacement  (MR) ,  the  complete  pair 
variable  only  (PP) ,  and  the  regression  replacement  (RG) .  The 
complete  observations  (CD)  method  will  also  be  included. 

The  major  purpose  of  the  study  is  to  compare  the  direct 
and  indirect  methods  in  their  ability  to  estimate  missing 
values,  variance-covariance  Z,  and  regression  parameters 
when  some  data  are  missing  in  both  independent  and  dependent 
variables.  The  study  will  include  several  factors:  degree  of 
skewness  of  the  data  (SK) ,  average  intercorrelation  of  the 
variables  (R) ,  percent  of  missing  entries  (PM) ,  type  of 
missing  data  (TM) ,  residual  variance  (E) ,  sample  size  (N) , 
and  number  of  variables  (P) . 

Significance  of  the  Study 
The  studies  reported  by  Beale  and  Little  (1975)  and 
Little  (1979)  point  to  the  superiority  of  the  direct  methods 
when  regression  parameters  are  calculated  from  estimated  V-C 
matrices.  Brown  (1983)  reported  similar  results  for  factor 
analysis.  However,  all  three  studies  used  only  multivariate 
normal  data.  Azen,  van  Guilder,  and  Hill  (1989)  used 
estimated  regression  parameters  to  compare  direct  and 
indirect  methods.  Besides  assuming  that  data  were  missing 


completely  at  random,  Azen  et  al.    (1989)   also  assumed  that 
(a)  the  missing  values  occurred  only  in  the  independent 
variables,    (b)  the  intercorrelations  between  any  pair  of 
variables  were  the  same,  and  (c)  the  errors  were  normally 
distributed.  To  date,  no  study  has  looked  at  the 
characteristics  of  the  estimated  V-C  matrix  and  regression 
parameters  when  (a)  data  from  both  independent  and  dependent 
variables  are  missing,    (b)  the  intercorrelations  between 
pairs  of  variables  are  not  the  same,  and  (c)  the  data  are 
from  a  skewed  distribution,  a  common  characteristic  of  data 
from  education  research.  The  aim  of  this  study  is  to  meet 
these  shortcomings  in  the  literature.  Of  particular 
importance  is  the  possibility  that  indirect  methods  may 
perform  nearly  as  well  as  direct  methods  when  the  normality 
assumption  is  violated.  Despite  the  superiority  of  the 
direct  methods,  researchers  may  tend  to  avoid  them  because 
of  complexity. 

Notation 
The  notation  used  is  as  follows: 

1.  Bold-faced  capital  letters  designate  matrices,  for 
example  AxB  means  multiplication  of  matrices  A  and  B. 

2.  Bold-faced  small  letters  designate  vectors. 

3.  Letters  in  ordinary  type  designate  a  cell 
observation  or  a  scalar  depending  on  the  text. 


Furthermore,  all  abbreviations  used  are  summarized  in  the 
Appendix,  Table  28. 


CHAPTER  2 
REVIEW  OF  THE  LITERATURE 


Least  Squares  Estimation 
Let  X  be  an  Nxp  data  matrix  where       is  the  value  for 
observation  i  on  variable  j;  i=l,...,N;  j=l,...,p.  Let 
y=Xp+i,  such  that  X*=(X,y),  of  dimension  Nx(p+l)  ,   is  the 
combined  data  set.  Suppose 

^yk  ^p)=Po^EPj^,- 

var{y\x^  ,Xp)=a^ 

and  x,,...,Xp,y  have  a  joint  distribution  with  mean 
H={Hi,  . . . ,  fjLp,  fiy)  and  V-C  matrix 


XX  xy 

S  a 

yx  ^yy. 


(2) 


where  2:„  is  the  (pxp)  V-C  matrix  of  the  independent 
variables  (Xj,   i=l,...,p),     1L^=Y,^^'  is  the  (pxl)  column 
vector  of  covariances  of  the  dependent  variable  (y)  and  each 
of  the  independent  variables,  and        is  the  variance  of  the 
dependent  variable.  Standard  regression  theory  gives 
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and 

To  estimate  the  parameters,  the  data  are  reduced  to  the 
sample  means 

jc=(3ci,  ^p'y=^p*i) 

and  the  sample  V-C  matrix  S=(Sj^) 

N 

i-i 

In  the  absence  of  missing  values,  the  multiple  regression 
parameters  can  be  estimated  from  the  estimate  of  the  mean 
vector  and  V-C  matrix. 


Maximum  Likelihood  Estimation 
Let  X  (Nxp)  be  a  data  matrix  representing  an 
independently  and  identically  distributed  sample  of  N 
observations  from  the  multivariate  normal  distribution  with 
mean  vector  /i=  (/iii ,  .  .  .  , /Ltp)   and  V-C  Z={aj,(,   j=l,  .  .  .  ,p; 
k=l,...,p}.  The  density  of  X  is 


f(2r|ti,S)=(27i)  2exp 


N 
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where  |£|   denotes  the  determinant  of  2.  Thus,  the  likelihood 
L(e|X)   of  e=(M,  Z)   is  a  function  of  ^i  and  Z  for  fixed 
observed  X.  The  ML  estimate  of  0  is  a  value  of  0  that 
maximizes  L(0lX)   or  equivalently  the  natural  logarithm  of 
the  likelihood.  In  L(0 | X) =1 (0 | X) .  For  many  models,  the  ML 
estimate  is  unique  and  furthermore  the  likelihood  function 
itself  is  different iable  and  bounded  above.  In  such  cases, 
the  ML  estimate  can  be  found  by  differentiating  the 
likelihood  (or  the  loglikelihood)  with  respect  to  each 
component  of  ©,  setting  the  result  equal  to  zero,  and 
solving  for  0;  that  is 

a/a0i{l(0|X)  }=0;  i=l,...,k 
where  k  is  the  number  of  components  in  0.  Thus,  the 
likelihood  equation  gives  a  set  of  k  simultaneous  equations, 
defined  by  differentiating  1(0|X)  with  respect  to  all  k 
components  of  0.     For  the  multivariate  normal  sample,  the  ML 
estimates  of  n  and  Z  are 

(i=x=(3q  

Let  g(0)   be  a  one-one  function  of  the  parameter  0.  Then  the 
ML  estimate  of  g(0)   is  g(0),  the  function  evaluated  at  the 
ML  estimate  0  of  0.  The  observed  information  matrix  is 
defined  by 

i(0|x)=ava0'{i(0|x)}|e=; 


and  the  standard  errors  of  the  estimates  are  given  by  the 
diagonal  elements  of  (I(e|X))''. 

In  the  presence  of  missing  values,  the  likelihood 
function  will  depend  on  whether  the  missing  data  mechanism 
can  be  ignored  or  not  (Little  &  Rubin,   1987;  Rubin,  1976). 
Little  and  Rubin  (1987)  said  that  if  knowledge  of  the 
mechanism  that  leads  to  the  missing  data  values  does  not 
influence  the  statistical  model,  then  the  mechanism  is 
ignorable;  otherwise  the  model  will  include  the  mechanism  by 
including  a  distribution  for  the  response  indicator. 

Distinction  and  Relevance  of  Missing  Data  Mechanism 
In  a  single  case  i,  i=l,...,N  suppose  X;  (1  x  p)   is  a 
vector  of  p  variables.  Suppose  Xj  is  partitioned  such  that 
Xi=(Xio,Xi„)  where  Xjq  (Ixk)   is  a  vector  of  k  (k<p)  observed 
variables,  and  Xj„  (Ix(p-k))   is  a  vector  of  missing 
variables.  Similarly,  suppose  X=(Xo,X„),  an  Nxp  data  matrix 
has  a  density  function  f(X|0),  where  ©  is  the  parameter 
vector  of  the  data.  The  density  function  for  Xq  is 

f  (Xo|©)=if  (Xo,X„|0)dX„ 
The  likelihood  of  0  based  on  Xq  ignoring  the  missing  data 
mechanism  is  a  function  proportional  to  f(Xo|0): 

L(0|Xo)   a  f(Xo|0) 
Little  and  Rubin  (1987)  pointed  out  that  knowledge  of  the 
mechanism  that  leads  to  the  missing  data  values  is  a  key 
element  in  choosing  an  appropriate  analysis  and  in 
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interpreting  the  results.  Where  the  mechanism  leading  to  the 
missing  data  does  not  enter  explicitly  in  the  data  analysis, 
the  mechanism  is  assumed  ignorable  and  ML  estimation  is 
based  on  L(©|Xo);  otherwise  the  likelihood  should  include 
the  missing  data  mechanism  by  including  a  distribution  for 
the  response  indicator.  Let  I  be  the  response  indicator  such 
that  1=1  if  an  item  is  observed  and  1=0  otherwise.  Thus  the 
data  consist  of  (Xo,I).  Suppose  the  conditional  distribution 
of  I  given  X  is  indexed  by  an  unknown  parameter  ^.  By 
definition 

f  (Xo,l|0,*)=Jf  (Xo,X„|©)f  (l|Xo,X„,^)  dX„ 
The  likelihood  of  the  parameters  is 

L(e,^\X^,l)  a  f(Xo,l|0,^)  (3) 
Rubin  (1976)  defined  missing  at  random  (MAR)  and  missing 
completely  at  random  (MCAR)  as  follows:  If  the  probability 
of  response  is  independent  of  Xq  and  X„,  then  the  missing 
data  are  MCAR.  That  is, 

f  (l|Xo,X„,*)=f  (I|*) 
Thus,   if  the  data  are  MCAR  the  likelihood  of  the  parameters 
is 

L(©,^|Xo,I)  a  f  (Xo|0)f  (llSk) 
If  the  joint  parameter  space  of   (0,'^')   is  the  product  of  the 
parameter  space  of  each  separately,  that  is  the  parameters  0 
and  ^  are  distinct,  the  likelihood  of  0  is 

L(0|Xo,I)  oc  f(Xo|0)  (4) 
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Thus,  the  missing  data  mechanism  is  ignorable  for 
likelihood-based  inferences.  The  estimation  of  0  would  yield 
correct  inferences  from  just  the  observed  data  (Rubin, 
1976) . 

If  the  probability  of  response  depends  on  Xq  but  not  on 
then  the  missing  data  are  MAR.  That  is, 

f  (llXo,X„,^)=f  (l|Xo,*) 
Thus  the  likelihood  of  the  parameters  is 

L(0,^|Xo,I)  oc  f  (Xo|0)f  (l|Xo,^) 
In  addition,   if  the  parameters  0  and  ^  are  distinct,  then 
the  likelihood  of  0  is 

L(0|Xo,I)  a  f(Xo|0) 
Thus,  the  missing  data  mechanism  is  ignorable  for  the 
likelihood-based  inferences. 

If  the  probability  of  response  depends  on  X„,  the 
missing  data  are  neither  MCAR  nor  MAR.  The  missing  data 
mechanism  is  nonignorable. 

If  data  are  MAR  or  MCAR,  the  missing  data  mechanism  can 
be  ignored,  and  the  likelihood  function  of  0  is  based  on 
only  the  observed  data  equation  (4) .  If  the  missing  data 
mechanism  is  nonignorable,  then  the  likelihood  function 
should  include  a  missing  data  indicator,   I,  variable.  The 
likelihood  function  will  be  of  the  form  in  equation  (3) . 
When  the  data  are  MAR,  the  aim  is  to  maximize  the  likelihood 
L(0|Xo);  otherwise  L(0,*|Xo,I)   is  maximized. 
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Little  and  Rubin  (1987)  pointed  out  that  for  many 
patterns  of  incomplete  data  the  likelihood  function  does  not 
allow  explicit  ML  estimates.  In  such  cases,  ML  estimates  can 
be  obtained  by  iterative  methods.  One  method  that  has  come 
to  common  use  is  the  EM  algorithm. 

The  EM  algorithm  proposed  by  Dempster,   Laird,  and  Rubin 
(1977)  does  not  require  second  derivatives  of  the 
loglikelihood  function  to  be  calculated.  The  algorithm  is  an 
iterative  method  of  maximum  likelihood  estimation  that  can 
be  applied  to  any  pattern  of  missing  data  (Dempster  et  al., 
1977;  Liski,  1985).  Each  iteration  consists  of  an 
expectation  step  (E-step)  and  a  maximization  step  (M-step) . 
The  algorithm  begins  with  a  starting  value,  q'^  for  0  and 
replaces  the  complete  data  loglikelihood,   1(0|X),  with  its 
expectation  given  the  observed  data  and  0^°^,  E[  1  (0 1  X)  |  X^,  0°] . 
This  is  the  E-step.  The  M-step  involves  the  maximizing 
E[1(0|X)  |Xo,0<°>]  to  obtain  a  second  estimate  0^'>  of  0.  Each 
step  of  the  algorithm  increases  the  loglikelihood  of  0  given 
Xq.  The  procedure  is  repeated  until  convergence  occurs. 

Beale  and  Little  (1975)  have  outlined  the  steps  of 
finding  the  ML  estimates  of  0=(/i,2)  through  the  EM 
algorithm.  Let  V;  be  the  set  of  variables  observed  in 
observation  i,   i=l,...,N.  The  estimation  equations  are 


(5) 


•''1=1 


(6) 


(7) 


where 


Ci/>=cov(Xy,Xi,|Vi;M^\Z^'^) 
and  Xjj*^=Xij  if  x^  is  observed, 

=  a  linear  combination  of  the  variables  in  Vj,  if  Xy 


is  missing. 


The  equations  (5)  to  (7)  are  iterated  until  a  specified 
difference  between  successive  iterations  is  met;  the  ML 
estimates  are  thus  obtained. 

The  algorithm  has  slow  convergence  properties, 
particularly  when  the  amount  of  missing  data  is  relatively 
large.  Furthermore,  the  algorithm  does  not  provide  standard 
errors  for  the  ML  estimates.  However,   it  is  insensitive  to 
initial  value;  that  is,   it  converges  even  if  the  initial 
value  is  poor. 

Methods  that  Disregard  the  Incomplete  Observations 
There  are  three  methods  that  estimate  the  variance- 
covariance  matrix  and  the  regression  parameters  without 
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assuming  an  underlying  data  distribution  and  without 
estimating  the  missing  entries. 

Complete  Data  Only  (CD)  Method 

Only  cases  which  are  complete  in  all  p  variables 
Xi=(Xi,,...   ,Xip)   are  used  to  estimate  Z  and  the  regression 
parameters  through  least  squares.  Therefore,   if       and  Xj,, 
i=l,...,N,  are  any  two  components  of  X;,  then  for  j,k  = 
1,  ,p 

N 
N 

^k=—  

and 

N 

E  ^ii^ir^j){^ik-^k) 

where  Xi=l  if  X;  is  complete,  otherwise  X~0,  and  np=Si=,'^  \  is 
the  number  of  complete  cases. 

Advantages  of  this  approach  are  (a)  simplicity,  since 
standard  complete-data  statistical  analyses  can  be  applied 
without  modifications,  and  (b)  comparability  of  univariate 
statistics,  since  these  are  all  calculated  on  a  common  set 
of  cases.  The  major  disadvantage  is  the  potential  loss  of 
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information  in  discarding  incomplete  cases.   Suppose  PM  is 
the  proportion  of  all  the  data  in  X  that  are  missing. 
Gleason  and  Staelin  (1975)  have  shown  that  the  probability 
that  any  row  of  X  will  have  no  missing  entry  is 
approximately  (1-PM)p.  Thus,  as  the  number  of  variables,  p, 
increases  the  proportion  of  rows  with  complete  data 
decreases  rapidly. 

Complete  Pair  Variables  Only  (PP)  Method 

Glasser  (1964)  proposed  a  method  that  uses  all 
available  pairs  of  observations  on  each  pair  of  variables  to 
estimate  Z.  Consequently,  the  method  is  sometimes  known  as 
Glasser  method.  Using  the  notation  described  in  the  CD 
method  paragraph,  for  j,  k  =l,...,p, 

N 




N 

5^  ^ik^i 


^jk 


and 


N 


5^  ^iak){^ij  ^j){^ik  ^k) 


njk-^ 


where  Xj^j^  =  1  if       and  Xj^  are  present,  otherwise         =  0, 
and  njk  =  ^i=i^  \(jk)  is  the  number  of  cases  with  both  Xj  and  x^ 
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present.  The  major  disadvantage  of  the  method  is  that  the 
estimated  V-C  matrix  is  not  necessarily  positive  definite. 
This  problem  is  severe  when  the  variables  are  highly 
correlated  (Little,   1992) .  Haitovsky  (1968)   found  that  with 
highly  correlated  data,  the  PP  method  was  inferior  to  the  CD 
method.  Kim  and  Curry  (1977)  found  that  the  PP  method  was 
superior  to  the  CD  method  when  weakly  correlated  data  were 
used.  Azen  et  al.    (1989)   found  that  the  PP  regression 
estimates  were  inferior  to  the  ML  estimates  for  both  normal 
and  nonnormal  regressors. 

Allvalue  Method 

The  Allvalue  method  (Dixon,   1983)   is  very  similar  to 
the  PP  method.  The  two  differ  only  on  the  estimation  of  the 
mean  of  each  variable.  The  Allvalue  method  estimates  the 
mean  of  each  variable  using  all  observations  for  that 
variable  and  estimates  covariances  between  pairs  of 
variables  using  only  cases  complete  in  both  variables.  Using 
the  notation  described  in  the  CD  method  paragraph,  for  j,  k= 
l,...,p  the  covariance  between  variable  j  and  variable  k  is 
estimated  as  follows 

N 

X,  =  ^  
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N 



and 

N 


5^  ^iak){^ij  ^j){^ik  ^k) 


where  X-^g,)  =  1  if       and       are  present,  otherwise  \■^^^.^  =  0, 
and       =  Sj^,'^  Xjoj.)  is  the  number  of  cases  with  both  Xj  and  x^ 
present. 

Methods  that  Estimate  the  Missing  Values  First 
Wilks^  Method 

Wilks  (1932)  proposed  the  mean  replacement  (MR)  method: 
the  estimate  of  the  missing  values  on  variable  j  is  the  jth- 
variable  mean  from  all  nonmissing  values.  Little  (1992) 
called  this  method  the  unconditional  mean  imputation.  In 
regression  analysis,  Afifi  and  Elashoff  (1966)   called  such 
procedures,  based  on  the  substitution  of  means,   zero  order 
regression  (ZOR)  methods.  Little  and  Rubin  (1987)  pointed 
out  that  earlier  work  also  considered  the  case  where  the 
grand  mean  of  all  the  available  data  was  substituted  for  all 
missing  values.  Little  and  Rubin  (1987)  have  shown  that 
under  MCAR  the  variance  of  the  observed  and  imputed  values 
for  the  jth  variable  is 
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N-l 

where       is  the  estimated  variance  from  the  Nj  observed 
cases.  Thus,  the  sample  variance  from  the  filled-in  data  set 
underestimates  the  variance  by  a  factor  of  (Nj-l)  /  (N-l)  .  The 
sample  covariance  of  Xj  and       ( j<k=2 , . . . ,p)   from  the  filled- 
in  data   (Little  &  Rubin,   1987)  is 

N-l 

where 

and  Nj^  is  the  number  of  observed  pairs.  Again,  the  estimate 
from  the  filled-in  data  underestimates  the  magnitude  of  the 
covariance  by  a  factor  (Nj^-l)  /  (N-l)  .  Thus,   inferences  (tests 
and  confidence  intervals)  may  seriously  be  distorted  by  bias 
and  overstated  precision  (Little,   1992) . 

The  Classical  Least  Squares  fCLS)  Method 
For  the  multiple  regression  model 

p 

Yates  (1933)   estimated  /Xy/  /?=i8, ,  .  .  .  , ^p,  and  the  missing  data 
by  minimizing 
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where 


^  7=E  — 


n,.  is  the  number  of  vector  observations  that  are  complete. 
That  is,  all  observations  for  which  values  are  missing  are 
discarded  and  least  squares  regression  performed  on  only  the 
complete  observations.  The  estimators  of  /iy  and  /3  are  the 
same  as  those  in  the  CD  method.  The  missing  values  are 
estimated  by  using  the  estimated  model. 


The  Modified  Zero-Order  (MZOR)   and  the  First-Order 
Regression  (FOR)  Methods 

For  the  multiple  linear  regression  analysis 


the  modified  zero-order  method  (MZOR)   involves  minimizing 


/  \2  /  \2 

I  ^  1  I  P  » 


(Hill  &  Ziemer,   1983),  where  y=Xy.  is  the  jth  variable  mean 
calculated  using  cases  with  complete  data  only  and  Ij 
represents  the  index  set  of  cases  with  missing  observation. 
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Hill  and  Ziemer  (1983)  presented  the  first-order  regression 
(FOR)  method  which  involves  estimating  p  regressions 

k'l 

for  complete  observed  cases  only,  where  e,^  is  an  error  term. 
Least  squares  estimates  of  jS^j  from  complete  sample 
observations  are  obtained.  Then,  each  missing  entry  is 
replaced  by  its  predicted  value. 

Zero-Order   (ZOR)   and  First-Order  (FOR)   Combined  Method 

Jackson  (1968)   in  a  linear  discriminant  analysis  study 
employed  an  iterative  technique.  The  technique  is  to  regress 
any  variable  in  which  missing  observations  occur  on  all 
remaining  variables;  means  are  substituted  for  all  missing 
values  for  the  initial  iteration.  The  resulting  regression 
equations  are  used  to  estimate  the  missing  data.  The 
procedure  is  iterated  until  estimates  stabilize.  Jackson 
(1968)   found  that  this  procedure  was  superior  to  the  method 
of  substituting  means  for  the  missing  values. 

Two-Stage  Regression   (TSR)  Method 

Afifi  and  Elashoff   (1969a)   gave  a  two-stage  regression 
method  for  the  simple  linear  regression  method:  First,  the 
regression  of  x  on  y  utilizing  only  the  complete 
observations  is  estimated  by  least  squares.  From  the 
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estimated  equation,  the  missing  x  observations  are 
estimated.  Second,  the  regression  of  y  on  x  from  the 
completed  observations  is  estimated  by  least  squares.  From 
the  estimated  equation,  the  missing  y  observations  are 
estimated. 

Regression  Replacement  (RG)  Method 

Suppose  Xj,   i=l,...,N,   is  partitioned  by  columns 
(variables)   into  two  subvectors 

where  Xj  of  dimension  Ixq  (q<p)  contains  variables  on  which 
ith  observation  has  missing  data  and  X2  of  dimension  Ix(p-q) 
contains  the  variables  on  which  ith  observation  has  no 
missing  data.  Thus,  an  individual  case  is  partitioned  into 
an  observed  data  set  and  a  missing  data  set  and  X  is 
partitioned  conformably;  that  is, 

X=[X^  X^]  (8) 

Without  loss  of  generality,  assume  the  mean  of  Xj  is  zero 
and  var(Xj)=l  for  all  j=l,...,  p;  then  the  correlation 
matrix,  R,  is 


xix^ 

.^1 

Gleason  and  Staelin  (1975)  gave  the  regression  weights  for 
estimating  X,  from  X2  as  R22"'R2i.  Hence, 
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The  regression  method  requires  first  estimating  R  with  the 
CD,  MR,  or  PP  method.  The  missing  entries  for  any  individual 
(row)  of  Z  are  estimated  by  equation  (9)  with  the  nonmissing 
elements  of  the  row  becoming       and  the  matrices  R22  and  Rj, 
being  drawn  from  the  relevant  portions  of  the  estimated 
correlation  matrix.  When  the  CD  method  is  used  to  estimate 
R,   the  RG  method  is  called  Bucks  method.  Walsh  (1961) 
generalized  Buck's  method  by  computing  all  possible  linear 
regressions  of  the  variable  j  with  a  missing  value  on 
variable  k  for  j?*;,  j,k=l,...,p,  for  all  complete  pairs.  A 
weighted  average  of  the  estimated  values  was  proposed  to 
fill  the  missing  data. 

Combining  Buck's  and  Walsh's  ideas,  Afifi  and  Elashoff 

(1966)  proposed  use  of  all  possible  multiple  regressions  for 
the  estimation  of  the  missing  values.  However,  this 
technique  is  not  computationally  feasible  when  p  is  large 
and  a  large  proportion  of  the  data  is  missing.  Buck's  and 
Walsh's  methods  are  examples  of  what  Afifi  and  Elashoff 

(1967)  called  first-order  methods. 

Principal  Components   (PC)  Method 

Wold  (1966)  proposed  a  missing  value  estimation  method 
that  utilizes  the  principal  components  from  an  incomplete 
data  matrix.  Suppose  the  first  n  (n<p)   components  capture 
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most  of  the  error-free  information,  while  the  last  p-n 
components  reflect  mostly  the  effects  of  errors  of 
measurement.  The  method  is  sensitive  to  the  value  of  n 
considered  in  the  estimation.  The  theory  behind  this  method 
is  given  by  Gleason  and  Staelin  (1975) .  Applying  the 
Singular-Value  Decomposition  theorem,  X  can  be  written  in 
the  form 


where  D  is  a  diagonal  matrix  of  eigenvalues  of  the 
correlation  matrix  R  of  the  variables  of  X,  V  is  the  matrix 
of  eigenvectors  of  R,  and  D  =  XVD'"^.  Suppose  the  largest  n 
eigenvalues  of  R  are  retained.  Let  D„  be  the  (nxn)  submatrix 
of  D  which  contains  the  n  eigenvalues  and  V„  the  matrix  of 
corresponding  eigenvectors.  Then, 


of  rank  n  (Gleason  &  Staelin,   1975) .   Suppose  X  is 
partitioned  as  in  equation  (8) ;  and  correspondingly,  V„  is 
partioned  as  V„  =  [V,  Vj] ' ,  where  V,  is  qxn  and       is  (p-q)xn. 
Let, 


1 


X=UD  2  V' 


11 


12 


v^vi  v^vi  «i 


'22 


Gleason  and  Staelin  (1975)  have  shown  that 
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^=X,W',,(J-W1,)-^  (10) 
This  is  the  principal  components  method  for  reconstructing 
X,  from  X2.  The  Dear  method  (Gleason  &  Staelin,   1975)  uses 
only  the  first  principal  component  and  is  thus  a  special 
case  of  the  principal  component  method.  Gleason  and  Staelin 
(1975)   showed  that  the  covariance  matrix  of  X,  with  X,  is 

N-^xX=R^^-Rl^{I-ff^^)-^  (11) 

where 


R'= 


=R-VJ}VI, 


^11  ^12 

is  the  residual  matrix,  which  is  obtained  by  removing  that 
portion  of  R  which  is  accounted  for  by  V„  and  D„.  The  matrix 
R*  depends  on  the  p-n  smallest  eigenvalues  of  R.  From 
equation  (11)  ,   if  X  is  of  rank  n  or  less,  then  X,  reproduces 
Xj  exactly  provided  (a)  n<p-q  or  q<p/2  and  (b)  I-W,,  is 
nonsingular  (Gleason  &  Staelin,   1975) . 

Gleason  and  Staelin,    (1975)   studied  the  relationship 
between  the  RG  and  PC  methods  and  showed  that 

ff2i{^-ffii)'^=R^2R2i-R^2R^i{^-ff^^y^  ( 12 ) 

The  left-hand  side  of  equation  (12)   is  the  transformation 
used  by  the  Principal  Components  method,   equation  (10) .  The 
right-hand  side  is  the  transformation  used  by  the  Regression 
method,  equation  (9) ,  but  reduced  by  a  term  that  depends  on 
the  smallest  p-n  eigenvalues  of  R.  Thus,  theoretically  the 
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Principal  Components  method  and  the  Regression  method  are 
related  as  follows  (Gleason  &  Staelin,   1975) : 

1.  If  the  rank  of  X  exceeds  n,  then  R*  will  be  a 
nonzero  matrix.  Thus,  the  Principal  Components  method  will 
differ  from  the  Regression  method. 

2.  If  the  rank  of  X  is  less  than  or  equal  to  n,  then  R* 
and  thus  R'21  will  be  negligible  or  nearly  zero  matrix.  Thus, 
the  two  methods  should  perform  similarly. 

When  Is  the  CD  Method  Sufficient? 
Hill  and  Ziemer  (1983)  have  proposed  a  test  statistic 
for  checking  the  adequacy  of  the  CD  method  in  the  estimation 
of  regression  parameters  when  some  data  are  missing.  Let  Y 
and  X  be  partitioned  as  follows: 


where  X,,  (Jxp)  has  at  least  one  variable  missing  in  each  row 
and        (Jxl)   is  the  corresponding  observed  response  vector, 
X^  (nxp)   and  Y,  (nxl)   are  the  complete  data,  n=N-J.  For  the 
complete  sample  ^ 

Y^=Xji,^e^  (13) 

where       is  an  iid  random  variable  with  mean  0  and  variance 
a^.  The  LS  estimator  for  the  complete  sample  can  be  shown  to 
be 
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var(^cD)=^'(Xc'XJ-' 
/3cD  -  N(/3,a2(X/XJ-') 
For  the  incomplete  sample 

where  6,^  is  an  iid  random  variable  with  mean  0  and  variance 

.  Suppose  the  missing  entries  are  estimated  and  a  (Jxp) 
matrix  X„  is  substituted  for  the  incomplete       so  that 

y,  =  X,/?+X„/3-X„/3+e, 
=  X„/3+(X,-X„)/J+e, 
=  X„^+5+ek 

=  X„/3+v  (14) 
where  v=5+ek  and  E(v)=5.  Hill  and  Ziemer  (1983)  showed  that 
the  estimator  of  /3  that  combines  (13)  and  (14)  is 

They  also  showed  that 

var(3)=a2(zX-xX)"' 

and  that  the  matrix 

is  positive  semidef inite.  Furthermore,  they  showed  that  ^ 
was  biased,  while  jSco  was  not.  The  choice  of  /3cd  over  ^ 
depended  on  the  difference  of  the  mean  squared  errors  for 
the  parameter  estimates  (Hill  &  Ziemer,  1983) 
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MSE{^cd)-MSH^)=A 

where 
and 

Thus 

A  =  var(Pa,) -'i^ar(p) -(flias(0))2 

is  given  as 

A=(x/x,+x„'x„)-X'['^'(x„(Xc'xj-X'+i)-5«']x„(x/x,+x„'x„)-' 
which  was  shown  to  be  positive  definite  if 

2o2  2 

where  X  is  the  non-centrality  parameter  of  the  test 
statistic 

iYjc-XJco)'{^J,x!xyxi^fj-\Y^-Zj^) 

Which  is  reported  to  be  distributed  as  an  F(J,n-p,X)  if  the 
null  hypothesis 

Ho:X  <  1/2 

is  true,  where 


N-p 

From  this  analytic  approach,  Hill  and  Ziemer  (1983) 
concluded  that  the  CD  method  will  give  good  estimates  if  Hq 
is  rejected;  otherwise,  methods  that  estimate  missing  values 
first  should  be  used. 

Evaluation  of  the  Estimation  Methods 
Criteria  for  (a)  the  comparison  of  estimated  missing 
values  to  observed  values,    (b)  the  comparison  of  estimated 
V-C  matrix  to  Z,    (c)  the  comparison  of  estimated  V-C 
matrices  from  two  estimation  methods,    (d)  the  comparison  of 
estimated  multiple  regression  parameters  to  those  obtained 
from  the  complete  full  data,  and  (e)  the  comparison  of  the 
multiple  regression  model  fit  are  required  for  an  effective 
evaluation  of  the  estimation  methods. 

Evaluation  Based  on  the  Estimation  of  the  V-C  Matrix 

Timm  (1970)  found  that  criteria  based  on  determinants 
for  comparing  Z  to  t„  or  |z:„|/|t^|  for  comparing  t„  and 
Z^,  where  a  and  /3  are  any  of  the  missing  data  estimation 
methods,  are  heavily  biased  by  small  eigenvalues  since  the 
determinant  of  a  matrix  is  the  product  of  its  eigenvalues. 
Thus,  such  criteria  are  inappropriate.  Timm  (1970)  instead 
proposed  the  Euclidean  norm. 
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lS-2jp  =  tr((S-2.)(S-2„rt  (16) 
a  measure  of  closeness  between  the  estimated  and  the  true 
V-C  matrices.  The  major  drawback  with  the  use  of  the 
Euclidean  norm  is  that  its  distribution  is  unknown  and  thus 
one  cannot  formulate  a  statistical  significance  test.  Timm 
(1970)   suggested  the  following  rule:  choose  the  technique 
which  minimizes  E{||j:  -  He  further  suggested  the  ratio 

of  the  Euclidean  norms  as  a  measure  of  the  overall 
comparative  efficiency  of  any  pair  estimation  methods.  The 
efficiency  of  estimation  method  a  relative  to  method  /3  is 
defined  as 


eff 


Estimation  method  a  is  preferable  if  the  ratio  is  less  than 
one,  otherwise  method  /3  is  the  best. 

Gleason  and  Staelin  (1975)  modified  the  Euclidean  norm 
statistic  and  applied  it  to  the  correlation  matrix.  Their 


statistic  is  defined  as 


n  S}^]i  ■  (18) 


P(P-1) 


representing  the  root-mean-square  deviation  of  estimated 
versus  actual  values  for  the  off -diagonal  correlations. 

Another  way  of  comparing  the  estimation  methods,   is  to 
construct  a  measure  of  the  quality  of  the  estimated  missing 
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entry.  Let         be  the  value  in  a  complete  data  matrix  that 
corresponds  to  a  missing  entry  in  an  incomplete  matrix. 
Suppose  the  variance  of  variable  j  in  the  complete  data 
matrix  is  a^.  Let  x>^  be  the  estimate  of  Xjj  derived  by 
method  a.  Let  K  denote  the  set  of  entries  missing  from  the 
matrix.  Gleason  and  Staelin  (1975)  proposed  the  following 
statistic 

Q^=  V  (19) 

an  index  measuring  the  reconstruction  of  the  elements  in  K, 
where  PM  is  the  proportion  of  missing  values.  The  smaller 
Q„,  the  better  the  fit.  Therefore,  the  interest  is  to  find 
an  estimation  method  a  which  minimizes  Q„. 


Evaluation  Based  on  the  Multiple  Regression  Model  Fit 

Let         j=0,l,..,,p,  denote  the  multiple  regression 
coefficients  estimated  from  incomplete  data  by  method  a.  The 
estimated  coefficient  can  be  compared  to  the  true 
coefficient  by  a  t-test  (Little,  1979) 

B.-e . 

t  =_llZ__L2£ 

which  is  distributed  as  a  t  with  (N-p+1)  degrees  of  freedom. 

Beale  and  Little   (1975)   recommended  judging  an 
estimation  method  by  the  overall  success  of  the  regression 
equation  rather  than  by  closeness  of  individual  regression 
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coefficients  to  their  true  values.  Thus,  the  criterion  Beale 
and  Little  (1975)  proposed  was  the  residual  sum  of  squares 
of  deviations  of  the  observed  and  estimated  values  of  the 
dependent  variable  when  the  deleted  values  were  restored, 
defined  as 


1-1 


5a=E  yi-Po.-E  P..^. 


(20) 


where,         and         j=l,  ,p,  are  the  regression  coefficients 

estimated  by  using  method  a,  and  Xy  and  yj  are  the  values  of 
all  variables  from  the  complete  data  only;  and  n^  is  the 
number  of  complete  data  observations.  The  smaller  the  value 
of  S„,  the  better  the  estimation  method  a. 
Donner  and  Rosner  (1982)  computed 

rfi-=|Pir-Pi|i3i«-Pi|  (21) 
where        and        are  the  ith,   i=0,l,...,p  regression 
coefficient  from  methods  a  and  y  respectively,  and  jS;  is  the 
true  regression  coefficient.  Then,  Donner  and  Rosner  (1982) 
used  di„>0  as  a  measure  of  relative  accuracy  of  estimator 
to 

Donner  and  Rosner  (1982)  also  used  the  relative  squared 
errors  of  the  regression  coefficients  as  a  criterion, 
defined  as 
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^^3^  (22) 

i=0,l,...,p.   If  REi„<l,  Donner  and  Rosner  regarded  method  y 
as  better  than  method  a  for  estimating  jS;  . 

Azen  et  al.    (1989)  used  two  criteria  for  the  assessment 
of  the  estimation  methods: 

1.  A  measure  of  how  well  the  estimation  method 
estimated  the  regression  coefficients,  defined  as 


where 

given  no  missing  data,  and  p  is  the  number  of  independent 
variables. 

2.  A  measure  of  how  well  the  missing  values  are 


estimated,  defined  as 


q  =  — (24) 


mP^Yr  min  d? 


2 


where  the  summation  is  over  all  missing  values,  PM  is  the 
overall  proportion  of  missing  data  among  all  x^s, 

x'ij  is  the  estimated  missing  value,        is  the  true  value,  and 


min  di/=var(Xj) -C^C^-'C^, 
where         is  the  estimated  covariance  of  the  jth  variable,  x 
with  w  (the  vector  of  observed  variables  for  case  i  with  y; 
included)  ,  and         is  the  estimated  dispersion  matrix  for  W;. 
Then,  two-way  analyses  of  variance  of  the       and  values 
were  conducted  to  compare  the  estimation  methods. 

Comparison  of  the  Estimation  Methods  from  Past  Research 
The  comprehensive  review  by  Afifi  and  Elashoff  (1966) 
points  out  that  most  of  the  pioneering  work  dealing  with 
missing  data  techniques  in  the  estimation  of  Z  assumed  that 
the  data  were  from  a  multivariate  normal  distribution  and 
proceeded  to  obtain  ML  estimates  for  the  parameters. 
Comparative  studies  among  the  indirect  methods  have  been 
conducted  (Gleason  &  Staelin,   1975;  Timm,   1970).  However, 
similar  studies  involving  either  direct  methods  or  direct 
and  indirect  methods  are  very  scarce  and  incomplete.  In  thi 
section  a  review  of  what  has  been  done  so  far  is  presented. 

Comparison  Based  on  V-C  Estimation 

Timm  (1970)   compared  the  CD,  MR,  Dear,   and  Buck 
methods.  All  these  methods  are  indirect  methods.  The 
comparison  of  the  methods  was  based  on  the  accuracy  with 
which  Z  and  R  were  estimated.  From  educational  and 
psychological  research,  a  number  of  V-C  matrices  with  p=2,  I 
or  10,  and  average  intercorrelation  ranging  from  0.2  to  0.8 
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were  selected.  Complete  data  matrices  were  generated  by 
using  the  Kaiser  and  Dickman  (1962)  procedure.  Also,  sample 
size  (N=50, 100, 200) ,  and  percent  of  missing  data 
(PM=1,10,20)  were  varied.  Timm  (1970)  used  the  Kaiser  (1968) 
definition  of  average  intercorrelation 


where  X,  is  the  largest  eigenvalue  of  R.  From  each  of  the 
complete  data  matrices  a  fixed  proportion  of  entries  were 
deleted  at  random.  Then  each  of  the  estimation  methods  was 
used  to  reconstruct  the  complete  data  matrix,  and  its 
associated  V-C  and  R  matrices.  No  uniformly  best  estimating 
method  for  Z  was  found;  however,  based  on  efficiency  indices 
(equation  17),  along  with  the  variation  in  the  Euclidean 
norm,  Timm  (1970)   found  that 

1.  The  performance  of  the  various  estimation  methods 
depended  on  whether  Z  or  R  was  being  estimated.  Different 
conclusions  were  obtained  when  £  was  estimated  and  when  R 
was  estimated. 

2.  With  weakly  correlated  observations,  that  is  R<0.3, 
(a)  both  the  Buck  and  the  Dear  methods  were  recommended  for 
p>5,  N>100,   and  PM>1  for  the  estimation  of  Z;    (b)  avoiding 
all  methods  was  recommended  if  the  conditions  in  (a)  were 
not  obtained;    (c)  the  Buck  and  Dear  methods  worked  equally 
well  for  p>5,  N>100,  and  PM<10  for  the  estimation  of  R;  and 
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(d)  the  Dear  method  was  recommended  when  PM=20,  otherwise 
the  MR  was  recommended. 

3.  With  intermediate  correlations,  that  is  0.4<R<0.6, 

(a)  the  Buck  method  was  recommended  for  p>5,  N>50,  and  PM>1; 

(b)  the  Dear  method  was  recommended  for  p=2,  N>50,  and  PM>1 
or  for  p>5  and  PM=20.  Recommendations  (a)  and  (b)  were  made 
for  the  estimation  of  Z  and  the  estimation  of  R. 

4.  With  highly  correlated  observations,  that  is  R>0.8, 
the  Buck  method  was  recommended  for  p>5,  N>50,  and  PM<1,  for 
the  estimation  of  R,  otherwise  the  Dear  method  was 
recommended. 

5.  In  the  estimation  of  R,  Timm  (1970)  noted  that  both 
the  MR  and  CD  methods  tended  to  reduce  the  average 
intercorrelation  of  variables,  while  the  Dear  method 
increased  it.  The  Buck  method  reduced  it  only  under  low 
intercorrelation . 

Gleason  and  Staelin   (1975)   compared  the  PC,  Dear,  RG, 
PP,  and  MR  methods  in  terms  of  ability  to  estimate  the  R,  as 
well  as  in  terms  of  ability  to  estimate  missing  entries. 
Gleason  and  Staelin  (1975)  generated  complete  data  matrices 
for  combinations  of  sample  size  (N=50,   100,  200),  number  of 
variables  (p=10,   15,   30),  and  average  intercorrelation 
(R=0.1  to  0.6).  By  varying  percent  of  missing  data  (PM=4, 
12,   20) ,   three  incomplete  data  matrices  were  constructed  by 
random  deletion  from  each  complete  data  matrix.  Then,  each 
of  the  estimation  methods  was  used  to  estimate  the  missing 
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data  and  an  estimate  of  R  was  obtained.  Gleason  and  Staelin 
(1975)  defined  the  average  intercorrelation  as 

r^J  \\Rf-p]^  (26) 

Ip(p-i); 

where 

i-1 

and  Xj,   i=l,    .   .        p  are  the  eigenvalues  of  R.  Euclidean 
norm  statistics  (D„)  defined  as  in  (18)  with  R  matrices,  and 
the  index  of  missing  data  reconstruction  (Q„)  defined  as  in 
(19)  were  regressed  against  N,  p,  PM,  R  using  multiplicative 
regression  models  of  the  form 

Z=P oil -R)^'N^'p^'{PM)^*  (27) 

or 

Z=PoR^'N^'p^HPM)^*  (28) 

where  z  is  either  D„  or  Q„.  The  use  of  either  (27)  or  (28) 
was  justified  because  theoretically  the  following  conditions 
hold: 

(i)  -  0  as  R  ^  0 

(ii)  Dpp      0  as  R  1 

(iii)  D^jjj      0,  Dpp  -+  0  as  PM  -  0 

Thus,  equation  (28)  was  applied  to  the  MR  method,  while 
equation   (27)  was  applied  to  the  PP,  PC  and  RG  methods.  The 
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comparisons  of  the  estimation  methods  were  based  on  these 
regression  models.  Gleason  and  Staelin  (1975)   found  that 

1.  The  choice  of  the  estimation  method  for  R  was 
strongly  influenced  by  N  and  R.  When  there  is  little 
intercorrelation  in  the  data  and  few  observations,  the  MR 
method  is  best.  For  large  correlations  and  a  large  number  of 
observations,  the  PP  method  performed  better  and  the  RG 
method  and  the  PC  method  performed  much  better  than  the  MR 
method.  For  example,  the  PP  method  was  better  than  the  MR 
method  for  N=50,   100,   200  when  R>0.25,   0.20,  0.15 
respectively. 

2.  For  values  of  R>0.2,  the  PC  and  RG  methods  gave 
estimates  of  the  missing  data  that  were  superior  to  those 
given  by  the  MR  method.  In  addition,  the  relative  gains  of 
the  PC  and  RG  methods  increased  rapidly  as  R  increased, 
irrespective  of  the  levels  of  PM  and  N. 

3.  For  R<0.2,  the  MR  method  was  best  since  there  was 
insufficient  redundancy  in  the  data  to  provide  any  better 
estimates  of  the  missing  entries  through  the  PC  or  RG 
methods  than  was  given  by  the  mean  of  the  variable.  This 
result  was  true  irrespective  of  the  levels  of  PM  and  N. 

4.  The  PC  method  was  computationally  faster  than  the  RG 
method;  but  the  two  methods  were  very  similar  when  their  Q„ 
and  D„  curves  were  studied. 

5.  No  broad  conclusions  were  given  in  relation  to 
either  the  percent  missing  data  or  the  number  of  variables. 
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However,  Gleason  and  Staelin  (1975)  acknowledged  that  the 
best  estimation  method  is  a  function  of  PM  and  p  as  well. 

Lee  and  Chiu  (1990)   compared  the  ML  and  PML  direct 
methods  and  the  CD,  MR,  RG,  and  PC  indirect  methods  for 
estimating  a  polychoric  correlation  when  data  are  missing. 
Discrete  trivariate  polytomous  observations  were  obtained  by 
transforming  trivariate  normal  variables  using  preassigned 
thresholds.  Some  of  the  observations  on  the  three  variables 
were  purposely  removed  at  random.  For  the  MR,  RG,  and  PC 
methods  cpmpleted  data  sets  were  obtained  and  the  parameter 
estimates  were  obtained  by  the  ordinary  ML  (OML)  method  for 
estimating  polychoric  correlations.  For  the  CD  method  OML 
for  estimating  polychoric  correlations  was  applied  to  the 
complete  data.  Also,  from  the  initial  data  before  missing 
entries  were  created,  OML  estimates  were  obtained.  Lee  and 
Chiu  (1990)   found  that 

1.  The  OML,  direct  ML  and  PML  estimates  of  the 
polychoric  correlation  matrix  were  very  close  to  each  other, 
and  the  ML  and  PML  estimates  were  generally  better  than  the 
estimates  produced  by  the  CD,  MR,  RG  and  PC  methods.  These 
results  were  more  pronounced  for  asymmetric  data  than  for 
symmetric  data.  Lee  and  Chiu  (1990)  did  not  work  with  data 
from  specific  known  distributions;  but  rather,  they  used 
data  distribution  pattern  to  classify  data  as  symmetric  or 
asymmetric.  However,  the  initial  correlation  matrix  used  in 
the  RG  and  PC  methods  was  calculated  using  CD,  which  may 
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have  affected  the  results.  Gleason  and  Staelin  (1975),  found 
that  the  initial  R  has  a  strong  influence  on  the  performance 
of  the  RG  and  PC  methods. 

2 .  The  direct  ML  and  PML  estimates  were  not  as  strongly 
affected  by  the  asymmetry  of  the  data  as  were  the  indirect 
methods . 

3.  For  the  direct  ML  and  PML  methods,  the  root  mean 
squared  error  associated  with  larger  population  correlations 
was  smaller  than  that  associated  with  smaller  population 
correlations. 

4.  The  estimates  obtained  by  using  the  MR,  RG,  and  PC 
methods  were  very  close  to  each  other.  The  CD  method  gave 
better  estimates  than  the  MR,  RG  and  PC  methods. 
Theoretically,  the  CD  method  ought  to  perform  less  well  than 
the  other  methods  because  as  noted  earlier,  as  the  number  of 
variables  (p)   increases,  the  proportion  of  rows  with  ■ 
complete  data  decreases  rapidly. 

5.  The  PML  method  was  comparable  with  the  direct  ML 
method  in  terms  of  accuracy.  However,  the  direct  ML  was 
recommended  because  it  resulted  in  accurate  estimates  with 
moderate  sample  sizes  even  when  there  were  many  missing 
observations . 

Comparisons  Based  on  Regression  Model  Fit 

Afifi  and  Elashoff   (1967)   studied  the  CD,  MR,   and  MZOR 
methods  for  estimating  jS  in  the  simple  linear  regression 
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model  when  the  missing  Y  and  X  observations  occured  at 
random. 

From  the  estimators  of  the  jSs,  Afifi  and  Elashoff 
(1967)   found  that 

1.  For  fixed  p   (correlation  of  y  and  x)  ,  m,/N,  my/N, 
where  m,  and  my  are  the  number  of  missing  observations  on  x 
and  y,  the  efficiencies  of  the  MR  and  MZOR  coefficient 
estimators  relative  to  that  of  the  CD  estimators  decreased 
as  N  increased.  They  defined  the  efficiency  of  method  a 
regression  coefficient  estimator  relative  to  method  y 
estimator  as  MSE  (^^) /MSE  (/5„)  . 

2.  For  fixed  N,  m,,  lUy  these  efficiencies  decreased  as  p 
increased. 

3.  If   |p|<0.30  and  N<70  the  MR  and  MZOR  estimators  were 
better  than  the  CD. 

4.  For  fixed  N  and  m^/N,  the  efficiencies  of  the  MR  and 
MZOR  estimators  relative  to  the  CD  estimator  increased  as 
ffiy/N  increased  for  |pj<0.30.  The  same  was  true  if  the  roles 
of  X  and  y  were  exchanged. 

Afifi  and  Elashoff  (1969a)  studied  the  FOR,  the  TSR, 
and  ML  methods,  and  all  the  methods  studied  by  Afifi  and 
Elashoff   (1967) .  Again,  the  simple  linear  regression 
parameters  /3,  and       were  studied.  Though  no  assumption  of 
the  bivariate  normality  for  x  and  y  was  required  to  define 
the  estimators,  the  assumption  was  essential  for  the 
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derivation  of  the  asymptotic  distributions  of  the 
parameters . 

Let       and  p^  be  the  proportion  of  the  x  and  y 
observations  missing,  respectively.  For  the  estimators  of  /S, 
Afifi  and  Elashoff   (1969a)   found  that 

1.  The  TSR  estimator  of  /3  was  better  than  all  other 
estimator  and  comparable  to  the  ML  estimator,  for  all  p,  p,, 
and  Py . 

2.  For  fixed  |p|   and  p^^,  the  asymptotic  efficiency  of 
the  TSR  j8  estimator  increased  as  Py  increased.  The  same  was 
true  if  the  roles  of  x  and  y  were  interchanged. 

3.  Based  on  finite  sample  efficiencies,  the  TSR 
estimator  of  /3  was  comparable  to  the  ML  estimator  unless 
|p|<0.1  for  all  N>200.  If  |p|<0.1,  the  ZOR  estimator  of  /3 
was  better  than  all  other  estimators  for  N<3  00. 

For  the  comparisons  among  the  estimators  of      ,  Afifi 
and  Elashoff  (1969a)   found  that 

1.  The  TSR  estimator  was  better  than  the  rest,  except 
the  ML  estimator,   for  all  values  of  p  if  'p^<O.A  and  Py<0.4. 

2.  Based  on  asymptotic  efficiency,  the  TSR  estimator 
and  ML  estimator  were  comparable. 

Afifi  and  Elashoff   (1969b)    in  a  follow-up  note, 
compared  the  small  sample  performance  of  estimators 
estimated  by  the  FOR  and  the  TSR  methods.  Bias  and 
efficiency  of  the  estimators  of  /3  and       were  studied.  It 
was  found  that 


1.  When  N=20  and  p,=Py=0.4,  the  TSR  regression  method 
generally  had  smaller  bias  than  the  other  methods. 

2.  The  ranking  of  the  estimators  based  on  small  sample 
efficiency  agreed  with  the  ranking  based  on  Afifi  and 
Elashoff  (1969a)  asymptotic  efficiency. 

Beale  and  Little  (1975)  compared  six  methods  for 
estimating  /3  when  data  are  missing  at  random.  Only  the 
multivariate  normal  case  was  investigated.  Data  values  from 
both  the  independent  and  dependent  variables  were  missing  at 
random.  The  six  methods  were 

1.  Ordinary  least  squares  on  complete  observations 

only. 

2.  Buck's  method. 

3.  Iterated  Buck's  method,  which  gives  ML  estimates. 

4.  Ordinary  least  squares  with  Y  present  after  fitting 
missing  values  of  the  independent  variables  by  ML  using 
independent  variables  only. 

5.  The  method  in  4,  but  with  missing  values  given 
fractional  weights. 

6.  The  method  in  5,  but  using  a  covariance  matrix  for 
all  the  variables  found  by  method  3,  to  find  the  fitted 
values  and  estimate  the  weights. 

The  residual  sum  of  squares  of  deviations  of  the 
observed  and  fitted  values  of  the  dependent  variable  as 
defined  in  (20)  was  used  as  a  criterion  for  judging  the 
effectiveness  of  each  method.  The  study  was  repeated  ten 
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times.  Overall,  Method  3  was  superior  to  all  other  methods. 
Method  1  was  even  inferior  to  Method  2.  Method  5,  an 
improvement  over  Method  4  in  some  situations,  was  inferior 
to  Method  1.  Method  6,  an  improvement  over  Method  5,  did 
quite  well  but  was  still  inferior  to  Method  3 . 

Little  (1979)   studied  four  methods  for  estimating  the 
multiple  regression  parameters,  0,  and       when  some  data  are 
missing  at  random.  Only  the  multivariate  normal  distribution 
was  investigated.  Normally  distributed  data  of  four 
Vcuriables  x,,  Xj,  Xj,  and  y  for  five  problems  were  generated 
of  the  form 

where  et~N(0,CTy^)  and  (x,,X2,X3)  '  ~  MN(0,r)  .  Six  deletion 
patterns  were  used  to  generate  missing  values.  However,  only 
the  independent  variables  were  allowed  to  have  missing 
values.  For  each  problem  and  deletion  pattern  50  data  sets 
were  generated.  The  four  estimation  methods  studied  were: 

1.  Ordinary  least  squares  on  the  data  before  deletion. 

2 .  Ordinary  least  squares  on  the  complete  observations 

only. 

3.  ML  estimation  via  the  Hartley  and  Hocking  (1971) 
method  of  scoring. 

4.  ML  estimation  via  the  EM  algorithm. 

For  each  method,  the  V-C  matrix  and  the  regression 
parameters  were  obtained.  Little  (1979)  compared  the  four 


methods  under  six  deletion  patterns  by  analyzing  (a)  the 
pivotals  of  the  regression  parameters,  defined  as 

■f^3j)  =  %l4  (29) 

and  (b)  the  mean  squared  deviation  of       from  its  true  value 
denoted  by  MSE(j5j)  .  For  each  problem  and  deletion  pattern, 
50  data  sets  were  generated.  From  the  pivotals,  the  means 
(MPQ)  and  mean  sum  of  squares  (MSPQ)  were  calculated,  and 
two-way  analysis  of  variance  on  MPQ  and  MSPQ  values  was 
conducted  with  the  six  deletion  patterns  and  five  problems 
as  factors.  The  analysis  of  the  MSPQ  showed  that 

1.  The  distribution  of  0^  is  not  normal  for  methods  2, 
3  and  4. 

2 .  The  accuracy  of  the  normal  approximation  was  less 
satisfactory  for  the  severely  deleted  patterns  than  for  the 
less  severely  deleted  patterns. 

3.  When  deviations  of  the  MPQ  from  zero  were  studied, 
methods  3  and  4  had  slight  positive  bias. 

With  MSE(iSj)   values,   Little   (1979)   found  that 

1.  For  the  severely  deleted  patterns,  the  relative 
efficiency  of  method  1  to  either  method  3  or  method  4, 
defined  as  the  ratio  of  the  two  MSE(iSj),  was  very  low. 

2.  Methods  3  and  4  were  more  efficient  than  Method  2. 
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Donner  and  Rosner  (1982)   studied  four  missing  data 
estimation  methods  applied  to  multiple  regression  with  two 
independent  variables, 

yi=/3o+i8,x,i+^2X2i+ei;  i=l,...,N 
where  the  e,s  are  iid  with  mean  zero  and  variance  a^.  The 
four  methods  were  the  CD,  PP,  ML  and  linear  prediction 
methods.  Missing  values,  for  PM=10,  25,  and  50,  occurred  at 
random  on  X2  only.  The  correlation  of  y  and  x,  was  fixed  at 
0.2.  The  correlations  of  y  and  Xj,  and  x,  and  Xj  were  0.2, 
0.5  or  0.8.  For  each  combination  of  p^,, ,  Py^,  p^^^  and  PM, 
N=100  pairs  (x,,  X2)  were  generated  from  a  bivariate  normal 
distribution  with       =       =  0,  cr,^  =       =  1. 

Donner  and  Rosner  derived  the. MSEs  of  the  regression 
coefficients  and  noted  that 

1.  The  MSE  of  ^1  for  the  PP  method  relative  to  the  MSE 
of  01  for  the  CD  method  was  high  when  the  sample  correlation 
(r)  between  x,  and  Xj  was  small. 

2.  For  estimating  /S,,  the  accuracy  of  the  linear 
prediction  method  relative  to  the  accuracy  of  the  CD  method 
was  high  for  large  r. 

For  estimation  of         Donner  and  Rosner  showed 
analytically  that  MSE(jS2)   for  the  PP  method  was  greater  than 
USE  ($2)  for  the  CD  method.  Also,  for  estimation  of  182, 
accuracy  of  the  linear  prediction  method  and  the  CD  method 
were  equivalent. 
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To  compare  estimators  in  their  monte  carlo  study, 
Donner  and  Rosner  (1982)  used  the  following  criteria: 

1.  A  difference  criterion:  dij=  |  ^i^-^;  |  -  | /5ij-/3i  |  ,  i=l,2; 
j=c,p,l  denotes  the  CD,  PP  and  linear  prediction  methods 
respectively,  and  m  denotes  the  ML  method.  The  proportion  of 
100  replications  with  dij>0  was  a  measure  of  relative 
accuracy  of  the  estimator       to  0^^. 

2.  The  relative  mean  squared  errors 

RE,j-  !  i=l,2;  j=c,p,l 

From  the  comparison  of  estimators  of  both  /3,  and        the  ML 
method  was  more  accurate  than  the  CD  and  PP  methods.  Except 
when  the  correlation  between  y  and  Xj  was  low  (0.2),  the 
parameter  estimates  produced  by  the  ML  method  were  more 
accurate  than  those  produced  by  the  linear  prediction 
method . 

Hill  and  Ziemer  (1983)   studied  four  estimation  methods 
for  replacing  missing  regressor  values  under  varying  degrees 
of  multicollinearity .  The  missing  values  estimation  methods 
were  the  CLS,   ZOR,  MZOR  and  FOR.  For  each  estimation  method, 
the  improvement  in  sampling  variance  given  by  (15)  was 
calculated.  Also,  the  bias  of  /3  introduced  by  using  the  data 
that  included  estimated  missing  values  was  calculated  by 
using 

bias(^)  =  (X/X,+X„X„')-X(X-X„) 
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Hill  and  Ziemer  (1983)  found  that 

1.  In  terms  of  bias  and  variance  reduction,  relative  to 
the  other  estimation  methods,  the  ZOR  method  gains 
significantly  with  increases  in  the  degree  of 
multicollinearity . 

2.  The  estimation  precision  of  the  FOR  method  decreased 
rapidly  as  the  degree  of  multicollinearity  became  acute. 

3.  Under  a  high  degree  of  multicollinearity,  the  ZOR 
method  performed  better  than  the  rest. 

4.  The  gains  from  replacing  missing  values  rather  than 
discarding  observations  with  missing  values  became  greater 
when  severe  multicollinearity  was  present. 

Azen  et  al.    (1989)  studied  the  CD,  PP,  and  ML  methods 
in  the  estimation  of  regression  parameters  and  missing 
values.  The  ML  method  was  implemented  by  the  EM  algorithm. 
They  varied  the  degree  of  missing  data  (PM=5,  and  25),  the 
distribution  of  the  independent  variables  (normal,  mixture 
of  normals,  and  lognormal) ,  patterns  of  incomplete  data 
(random,  related,  and  censored) ,  the  intercorrelation 
coefficient  (R=0.25  or  0.75),  and  the  multiple  correlation 
coefficient  (R^=0.5  or  0.9).  The  correlations  between  all 
pair  of  independent  variables  were  the  same.  They  assumed 
missing  values  occured  only  in  the  independent  variables. 
The  dependent  variable  was  defined  by  the  multiple  linear 
regression  of  the  form 

Yi  =  j8o+i3iXi,+j82Xi2+/33Xi3+ei;  i=l,...,N 
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where,  ^-=1,  j=0,l,2,3  and  6;  were  iid  N(0,a^).  Fifty  data 
sets,  each  comprising  a  100x3  matrix  of  N(0,1)  variables 
were  generated.  Then,  data  with  the  desired  distribution 
were  obtained  by  transforming  these  data  sets. 

Azen  et  al.   (1989)  used  two  criteria,   (23)  and  (24), 
for  the  evaluation  of  the  estimation  methods.  The  results 
were  stratified  by  the  distribution  of  the  independent 
variables  and  the  deletion  pattern.  Two-way  analysis  of 
variance  for  each  criterion  was  conducted  within  each 
stratum.  , 

For  the  estimation  of  the  regression  coefficients,  Azen 
et  al.    (1989)   found  that 

1.  All  three  estimation  methods  performed  well  for 
moderate  multiple  correlation  coefficients  (R^=0.5). 

2.  When  R^=0.9,  the  PP  method  was  generally 
significantly  inferior  to  the  other  two  estimation  methods 
at  a=0.05. 

3.  The  CD  method  performed  as  well  as  the  ML  method, 
particulary  when  data  were  censored. 

For  the  estimation  of  the  mean  vector  and  covariance 
matrix,  Azen  et  al.    (1989)   found  that 

1.  When  R^=0.5,   all  three  methods  performed  well. 

2.  When  R^=0.9,  the  CD  and  PP  methods  were  generally 
inferior  (P<0.05)  to  the  ML  method. 


Generally,  the  ML  method  performed  better  for  the 
random  and  related  patterns  of  incomplete  data  than  for  the 
censored  data. 

Summary 

In  summary,  past  research  on  the  estimation  methods  has 
shown  the  following: 

1.  Among  the  indirect  methods,  the  RG,  PC  and  PP  are 
comparable,  and  better  than  the  MR  and  CD  methods  in 
estimating  the  variance-covariance  matrix  (Gleason  & 
Staelin,  1975;  Timm,  1970). 

2.  Lee  and  Chiu,    (1990)   in  estimating  polychoric 
correlations  from  incomplete  data  found  that  the  direct 
methods  of  ML  and  PML  gave  better  estimates  than  the 
indirect  methods  of  CD,  MR,  RG,   and  PC.  However,  when  the 
underlying  data  distribution  was  asymmetric  the  superiority 
of  the  direct  methods  was  not  highly  pronounced. 

3.  When  estimating  regression  parameters  for  both  the 
linear  and  multiple  regression,  the  ML  procedure  produces 
better  estimates  than  any  of  the  other  methods.  In  the 
linear  regression  when  the  proportion  missing  x  values  or  y 
values  is  less  than  0,1,  the  CD  estimates  are  comparable  to 
the  ML  estimates.  In  the  censored  data  the  CD  estimates  are 
comparable  to  the  ML  estimates. 
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4.  In  estimating  parameters  of  the  simple  linear 
regression  model,  the  TSR  estimates  are  comparable  to  the  ML 
estimates. 

What  Does  Literature  Lack? 

The  literature  shows  the  superiority  of  the  direct  ML 
method  over  the  indirect  methods  (Lee  &  Chiu,  1990)  for 
estimating  various  parameters  of  the  regression  model.  For 
the  estimation  of  the  V-C  matrix  no  one  particular 
estimation  method  among  the  indirect  methods  consistently 
surpasses  the  rest  (Gleason  &  Staelin,  1975;  Timm,   1970) . 
However,  Timm  (1970)  and  Gleason  and  Staelin  (1975)  found 
that  as  the  average  intercorrelation  among  the  variables 
increased,  the  estimates  produced  by  the  PC  and  MR  methods 
improved  and  were  consistently  better  than  the  rest  of  the 
estimation  methods.  The  CD  method  outperformed  the  remaining 
methods  when  the  intercorrelations  among  the  variables  were 
low.  Lee  and  Chiu  (1990) ,  studying  estimation  of  polychoric 
correlations,  found  that  the  ML  method  had  little  advantage 
over  the  indirect  methods  when  the  data  were  asymmetric. 

No  single  study  for  the  comparison  of  the  direct  ML  and 
the  indirect  methods  in  the  estimation  of  the  V-C  matrix  is 
available  in  the  literature.  However,  Lee  and  Chiu  (1990) 
have  shown  that  the  direct  methods  are  generally  superior  to 
the  indirect  methods  in  estimating  polychoric  correlation 
matrix. 
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Among  the  investigations  that  focused  on  the  regression 
model,  only  Azen  et  al.    (1989)   included  nonnormal  data.  That 
study  had  at  least  three  limitations.  First,  in  each 
condition  the  correlation  between  each  pair  of  independent 
variables  was  constant  across  pairs.  Second,  only  the 
independent  variables  had  missing  data.  Third,   in  the  case 
of  censored  data  the  effect  of  left  and  right  censoring  was 
not  investigated.  Among  the  investigations  that  focused  on 
the  V-C  or  correlation  matrix,  only  Lee  and  Chiu  (1990) 
included  nonnormal  data  and  compared  direct  and  indirect 
methods,  but  they  studied  estimation  of  polychoric 
correlation  matrices. 


CHAPTER  3 
METHODOLOGY 

In  this  chapter,  the  design  of  the  study  and  the  data 
generation  procedure  are  described.  Four  indirect  estimation 
methods,  namely  the  CD,  MR,  PP,  and  RG,  were  compared  to  the 
direct  ML  method  when  some  data  were  missing  completely  at 
random  or  censored.  For  the  indirect  methods,  Z  and  jS  were 
estimated  by  using  ordinary  least  squares  after  the  missing 
entries  were  estimated.  For  the  direct  method,  estimation  of 
/3  was  by  least  squares  after  Z  was  estimated. 

The  selection  of  the  factors  and  levels  in  the  design 
were  based  on  the  review  of  the  relevant  literature  with  the 
aim  of  making  the  experimental  conditions  used  in  the 
simulation  similar  to  those  found  in  educational  research. 

Design 

The  factors  to  be  considered  in  the  study  were  the 
degree  of  data  skewness  (SK) ,  type  of  missing  data  (TM) , 
sample  size  (N) ,  number  of  independent  variables  (p) , 
percent  of  missing  data  (PM) ,  average  intercorrelation  of 
the  variables  (R) ,  and  residual  variance  (E) .  The  levels  of 
these  factors  are  given  in  the  following  paragraphs. 
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Degree  of  data  skewness  (SK) .  Lee  and  Chiu  (1990) 
showed  that  when  the  data  distributions  were  symmetric,  the 
direct  methods  gave  better  polychoric  correlation  estimates 
than  did  the  indirect  methods.  However,  the  superiority  of 
the  direct  methods  over  the  indirect  methods  was  negligible 
when  the  data  were  asymmetric. 

Kendall  and  Stuart  (1963,  p. 57)  reported  on  age  at  time 
of  marriage  for  over  300,000  Australians.  They  found  that 
the  data  distribution  had  skewness  and  kurtosis  of  2.0  and 
8.3  respectively.  Micceri  (1989)  investigated  the 
distributional  characteristics  of  440  achievement  and 
psychometric  measures.  In  terms  of  the  shape  of  the  data 
distribution,  Micceri  found  that  28%  of  the  distributions 
were  relatively  symmetric,  41%  were  moderately  asymmetric, 
and  31%  were  extremly  asymmetric.  Furthermore,   11%  of  the 
data  distributions  had  skewness  as  extreme  as  2.0. 

In  addition  to  generating  regressor  variables  from  the 
multivariate  normal,  regressor  variables  were  generated  from 
a  multivariate  lognormal  distribution  with  the  degree  of 
skewness  of  2.0.  By  calculating  exp(y),  with  y  normally 
distributed  with  mean  zero  and  variance  0.3  04,  lognormal 
data  with  skewness  2.0  and  kurtosis  10.9  were  constructed. 

Sample  size        .  Theoretically,  as  N  increases,  the 
stability  of  the  estimates  of  r  and  of  the  regression 
parameters  should  increase.  Hence,  the  CD  and  PP  methods 


ought  to  yield  progressively  better  estimates.  The  MR  method 
is  also  sensitive  to  N  since  the  efficiency  of  the  mean  of 
the  observed  data  will  increase  with  N.  The  effect  of  N  on 
the  RR  and  PP  methods  can  be  deduced  from  the  fact  that  the 
estimation  methods  utilize  the  correlation  matrix  which  is 
sensitive  to  an  increase  in  N.  As  N  increases,  the 
likelihood  function  improves;  this  implies  that  the  sample 
size  has  an  influence  on  the  direct  methods  as  well.  Timm 
(1970) ,  and  Gleason  and  Staelin  (1975)  noted  that  the 
performance  of  the  estimation  methods  was  strongly 
influenced  by  N.  The  levels  of  N  were  N=100  and  300. 

Number  of  independent  variables  (p) .  The  effect  of  the 
number  of  independent  variables  was  also  of  interest.  Data 
were  generated  using  p=3,  or  6  independent  variables.  This 
choice  is  consistent  with  the  range  of  variables  commonly 
studied  in  educational  research  (Algina  &  Oshima,  1990; 
Tang,  1990) . 

Percent  missing  data  (PM) .  To  simulate  the  incomplete 
data,     some  entries  were  deleted  from  the  complete  vector- 
valued  data.  Past  research  has  shown  that  when  PM>2  0  the 
estimation  methods  are  likely  to  perform  poorly  (Gleason  & 
Staelin,   1973;  Timm,   1970).  Azen  et  al.    (1989)   found  that 
both  the  indirect  (CD  and  PP)  and  direct  (ML)  estimation 
methods  performed  equally  well  when  the  percent  of  missing 
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data  in  the  independent  variables  was  5;  the  ML  method  had 
an  edge  over  the  other  two  estimation  methods  when  PM=25, 
although  the  CD  method  was  found  to  be  safe  and 
conservative.  The  levels  of  PM  were  5%  and  15%.  In  this 
study,  data  were  missing  from  both  the  independent  variables 
and  dependent  variable. 

Tvpe  of  missing  data  (TM) .  Missing  values  may  arise  in 
many  ways.  For  example,  the  missing  variables  may  be  (a) 
forgotten  and  thus  not  measured,   (b)  measured  and  the  values 
lost,  or  (c)  unobservable  through  design  considerations. 
Cases  like  (a)  or  (b)  often  give  rise  to  data  that  are 
missing  completely  at  random,  while  cases  like  (c)  give  rise 
to  censored  data.  Three  patterns  of  incomplete  values  were 
studied:  missing  completely  at  random,  censored  from  left, 
and  censored  from  right.  For  all  three  patterns, 
Nx(p+1) xPM/100  data  points  were  deleted. 

Average  correlation  among  independent  variables  fR) . 
Gleason  and  Staelin  (1975)  found  that  the  performance  of  the 
estimation  methods,  particularly  the  indirect  methods,  was 
strongly  influenced  by  the  intercorrelation  of  the 
variables,  R,   as  defined  by  equation  (2  6) .  The  RG  and  PP 
methods  were  reported  to  have  performed  well  when  the 
intercorrelation  was  high  (R=0.6);  while  the  MR  did  well 
when  the  intercorrelation  was  low  (0.1<R<0.3)    (Gleason  & 
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Staelin,   1975)  •  The  influence  of  R  on  the  direct  methods  is 
not  very  clear,  but  since  the  methods  utilize  the  likelihood 
function,  increase  in  R  ought  to  improve  the  estimation  of  Z 
and  /3.  Timm  (1970)   studied  data  sets  which  had  R  between  0.2 
and  0.8;  while  Gleason  and  Staelin  (1975)  varied  R  between 
0 . 1  and  0.6. 

This  index,  R,   is  highly  related  to  the  average  • 
correlation  (Gleason  &  Staelin,  1975)  .  In  this  study  the 
average  correlation  (r)  was  used  in  place  of  R.  Although  r 
ranges  between  0  and  1,  values  of  r>0.85  are  unrealistic  and 
may  cause  multicollinearity  of  the  variables.  Among  the 
consequences  of  multicollinearity  are  (a)  regression 
coefficients  tend  to  be  poorly  estimated  and  may  have 
incorrect  sign;   (b)  variances  associated  with  the  estimates 
of  the  regression  coefficient  may  be  greatly  inflated;  and 
(c)  predicted  values  may  be  unrealistic.  This  factor  r^,  from 
this  point  onwards  will  be  simply  denoted  as  R.  Thus,  the 
levels  of  R  were  0.2  and  0.7. 

The  correlation  matrices  R  used  in  the  study  are 
presented  in  Table  1.  The  important  features  of  these  R 
matrices  were  that 

1.  The  off-diagonal  elements  are  not  all  equal. 

2.  Each  matrix  has  a  single  dominant  principal 
component,   as  shown  by  the  eigenvalues. 

3.  All  the  eigenvalues  are  positive. 


Table  1 


Correlation  Matrices 


Number  of  variables:  p=3 


R= 


1.00  0.11  0.33 
0.11  1.00  0.23 
0.33  0.23  1.00 


Eigenvalues:  \  =1.4  6  0.90  0.64 

Range  of  pairwise  correlations:  0.11  -  0.33 

Average  correlation:  R=0.22 


Number  of  variables:  p=3 


R= 


1.00  0.64  0.73 
0.64  1.00  0.82 
0.73  0.82  1.00 


Eigenvalues:  \  =  2.46  0.37  0.16 

Range  of  pairwise  correlations:  0.64  -  0.82 

Average  correlation:  R=0.73 


Table  1 — continued. 


Number  of  variables:  p=6 

1.00  0.11  0.33  0.23  0.11  0.33 

0.11  1.00  0.23  0.11  0.33  0.23 

0.33  0.23  1.00  0.33  0.23  0.11 

0.23  0.11  0.33  1.00  0.11  0.33 

0.11  0.33  0.23  0.11  1.00  0.23 

0.33  0.23  0.11  0.33  0.23  1.00 

Eigenvalues:  X;  =2.12  1.09  0.89  0.77  0.67  0.46 
Range  of  pairwise  correlation:  0.11  -  0.33 
Average  correlation:  R=0.22 


Number  of  variables:  p=6 


1.00  0.64 
0.64  1.00 
0.82  0.73 
0.73  0.64 
0.64  0.82 
0.82  0.73 


0.82  0.73 
0.73  0.64 
1.00  0.82 
0.82  1.00 
0.73  0.64 
0.64  0.82 


0.64  0.82 
0.82  0.73 
0.73  0.64 
0.64  0.82 
1.00  0.73 
0.73  1.00 


Eigenvalues:   \  =  4.65  0.53  0.36  0.27  0.18  0.01 
Range  of  pairwise  correlations:  0.64  -  0.82 
Average  correlation:  R=0.73 
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Conditional  variance  (E) .  Conditional  (residual) 
variance  was  set  at  two  levels,  E=6.0  or  12.0.  These  levels 
were  chosen  so  that  the  coefficient  of  determination  varied 
between  0.25  and  0.80. 

Simulation  Procedure 
The  initial  factors  included  sample  size,  number  of 
variables,  degree  of  data  skewness,  average  correlation 
among  independent  variables,  and  residual  variance,  all  at  2 
levels  giving  32  conditions.  The  simulation  was  conducted  as 
32  separate  runs,  one  for  each  condition,  with  50 
replications  per  condition;  giving  the  full  complete  data. 
Then,  for  each  condition  5%  or  15%  of  the  data  were 
independently  deleted  at  random  (RM)   or  left  censored  (LC) 
or  right  censored  (RC) ,  giving  data  with  missing  values  for 
192  total  conditions.  For  each  condition  the  performance  of 
the  five  estimation  methods  were  evaluated. 

A  Multivariate  Loqnormal  Distribution 

Suppose  Y  is  a  p-variate  normal  distribution  with  means 
Mi,  variances      ,  and  correlations         for  i,j=l,...,p.  Let 
x=(Xi,  .  .  .  ,Xp) '  =  [exp(y,)  ,  .  .  .  ,exp(yp)  ]  '  .  Jones  and  Miller  (1966) 
showed  that  X  has  a  p-variate  lognormal  distribution  with 
probability  density  function  given  by 
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f{X)=  ^-^  exp/l(logX-ti)'S-i(logX-ti)j 


for  Xi>0,  i=l,...,p  and  f(X)=0  otherwise,  and  means, 
variances,  and  correlations  defined  as 


of =exp(2  n  j +2  Oj)  -exp(2  +0^) 


/  exp(Q,.,.o,o^.)-l 


2 


(expoj-l)  2(expaj--l) 
In  the  univariate  case,  skewness  and  kurtosis  are  defined  as 


3  3 


respectively.  For  the  lognormal  distribution  Mood  et  al, 
(1974)  have  given  the  rth  moment  about  the  origin  as 


p^=£(x^)=exp 


Thus,  the  third  central  moment  of  a  lognormal  variable  is 

/i3'=E[  (x-m')^]=E(x3)-3m'E(x2)+3)U'2E(x)-m'^ 
that  is. 
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=exp|3  n  + 1     -3  exp|3  n  + 1  o^j +2  expjs  + 1  o'j 

and 

2 

o^'=(exp(2^+2o^)-exp(2n+a2))  ^ 
The  skewness  of  a  lognormal  variable  is 

exp|3   +  ^     -3  expjs   +        +2  exp|3  p  +  ^  a^j 

(exp(2ji+2o2)-exp(2n+a2))  ^ 
The  fourth  central  moment  of  a  lognormal  variable  is 

that  is, 

H4  =exp(4  |i  +8  0^)  -4  exp(4  n +5  a^)  +6  exp(4  n  +3  a^)  -3  exp(4  +2  o^) 

and 

o^'=(exp(2  \i  +2  o^)  -exp(2  n +02))^ 
Thus,  kurtosis  of  a  lognormal  variable  is 

p  _  exp(4 n + 8 g^) -4 exp(4 \i+5a^)+6 exp(4 \i+3o^) -3 exp(4 \i+2a^) 

(exp(2  \i  +2  o^)  -exp(2  n  +0^))^ 

Following  Johnson  (1987),  /x  can  be  set  to  zero  since  it  acts 

as  a  scale  parameter  for  the  lognormal  distribution.  With 

this  specification  jSi  and       are  determined  by  a^.  Selecting 

a^=0.304  gives  the  j8,=2 . 0  and  182=10.9. 
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Generation  of  the  Data 

Multivariate  normal  regressor  variables  were  generated 
by  using  the  following  steps: 

1.  By  p  successive  calls  to  a  univariate  normal 
generator, 

z  -  Np(0,I)  was  generated. 

2.  The  pxl  vector  x  was  calculated  by  using  x=Az  where 
AA'=Z,  r=DRD,  R  was  one  of  the  specified  correlation 
matrices.  Table  1,  and  D  was  a  pxp  diagonal  matrix  with  D~a^ 
where  ai^=0.304.  The  distribution  of  x  is  Np(0,Z)  . 

The  multivariate  lognormal  regressor  variables  were 
generated  by  using  the  following  steps: 

1.  By  p  successive  calls  to  a  univariate  N(0,1) 
generator,   z  ~  Np(0,I)  was  generated. 

2.  The  pxl  vector  y  was  calculated  by  using  y=Az,  where 
AA'=Z.  The  distribution  of  y  is  Np(0,2:). 

3 .  The  pxl  vector  x    was  calculated  by  using 
x=(x,,  .  .  .  ,Xp)  '  =  [exp(y,)  ,  .  .  .  ,exp(yp)  ] ' 

The  matrix  S  in  step  2  was  based  on  the  following  results. 
As  noted  earlier  the  mean  and  variance  of  a  lognormal 
variable  x=exp(y)  are 


and 


65 


a\'=exp{2  n  j+2  Oj)  -exp(2    +a  j) 


where  /ij,  o^^  are  the  mean  and  variance  for  the  normal 
variable  y.  Johnson,  Ramberg,  and  Wang  (1982)  showed  that 


where  p^'  is  the  correlation  between  Xi=exp(yi)  and  Xj=exp(yj) 
and       is  the  correlation  between  y;  and  yj.  For  all 
conditions  n—O  and  tTi^O.304.  The  parameter  pjj'  was  the  ij  th 
element  of  the  correlation  matrix  (R)   in  Table  1.  Using  the 
relationship  between  p^  and  p,^' ,  the  correlation  matrix  R* 
was  calculated.  The  matrix  Z  was  then  calculated  as  Z=DR'D 
where  D—a^. 

4.  Standardize  the  lognormal  variable  so  that  it  has 
mean  zero  and  variance  ai^=0.304. 

For  the  regression  analysis,  Xp+i  =  y  variable  data  were 
generated  by  pre-specif ying  the  regression  model 

yi  =  /3o+/3,x,+  . .  .+^pXp+ei  ;  i=l,...,N 
where,  j8o=/3i= .  .  .  =j8p=l .  The  variable  e  was  iid  normally 

distributed  with  mean  zero  and  variance  a/  =  6.0  or  12.0. 

/ 

Selection  of  Randomly  Missing  Data 

To  select  PM  values  to  delete  from  an  Nx(p+1)  complete 
data  set,  these  steps  were  followed: 

1.  Generate  a  (0,1)  random  number. 
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2.  Multiply  the  generated  number  by  ( (Nx (p+1) ) -1)  and 
add  1  to  generate  a  random  number  in  (l,Nx(p+l)). 

3.  Round  the  number  in  step  2  to  the  nearest  integer. 

4.  Delete  the  observation  in  the  cell  number  given  in 
step  3,  counting  along  the  rows  of  the  complete  data  matrix. 

5.  Repeat  steps  1  to  4  Nx(p+1) xPM/100  times  to  delete 
Nx(p+l)xPM  missing  values.  In  the  event  the  same  cell  was 
identified  more  than  once,  steps  1  to  4  were  repeated  a 
sufficient  number  of  times  to  delete  Nx(p+l)xPM  missing 
values.  In  addition  a  record  with  all  variables  missing  was 
disallowed. 

Selection  of  Censored  Missing  Data 

Missing  values  were  censored  from  either  the  left  or 
the  right  of  the  data  distribution.  To  identify  data  points 
for  censoring,  the  following  steps  were  used: 

1.  From  the  data  set  with  randomly  missing  values,  the 
number  of  missing  values  nij  in  variable  j,  j=l, . . . ,  (p+1)  was 
known. 

2.  From  the  complete  data  set  for  the  jth  variable 
( j=l, . . . ,p+l) ,  delete  either  the  smallest  mj  values 
resulting  in  left-censored  data  or  the  largest  mj  values 
resulting  in  right-censored  data. 
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Evaluation  Techniques 


The  estimation  methods  were  evaluated  based  on  accuracy 
of  estimation  of  (a)  missing  values,    (b)  V-C  matrices,  and 
(c)  the  multiple  linear  regression  parameters. 

Evaluation  Based  on  the  Estimation  of  Missing  values 

Gleason  and  Staelin  (1975)  defined  the  index  of  data 
reconstruction , 


Estimation  is  better  when  Q„  is  smaller. 

Evaluation  Based  on  the  Accuracy  of  the  V-C  Estimates 

The  Gleason  and  Staelin  (1975)  modified  Euclidean  norm 
for  the  V-C  matrix  is  defined  as 


The  estimation  method  with  the  smallest  D„  is  prefered.  For 
the  comparison  of  two  estimation  methods,  Timm  (1970) 
defined  relative  efficiency  of  method  a  to  method  /3  as 


The  estimation  method  a  was  preferable  if  the  ratio  was  less 
than  one,  otherwise  method  |8  was  the  best. 


(30) 


(31) 
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Evaluation  Based  on  Estimation  of  Regression  Parameters 

Two  criteria  for  accuracy  of  estimated  regression 
parameters  were  employed: 

1.  Let        denote  the  estimate  of  /3j  calculated  by 
applying  method  a  and  let  jSj  denote  the  estimate  of  /3j 
calculated  by  applying  ordinary  least  squares  to  the  data 
before  deletion  of  the  data.  The  criterion  was 

t{h--hr  (33) 

2.  The  multiple  linear  regression  model  fit,  defined  as 


 2=1  , 


(34) 


where        and         j=l,...,p,  were  the  regression  coefficients 
estimated  by  method  a.,  x^-  and  yj  were  the  true  values  of  all 
variables  before  data  deletion,  and  N  was  the  sample  size. 
Estimation  is  better  when  s^„  is  smaller.  The  quantity  in 
equation  (34)   is  an  estimate  of  a^.  Thus,  the  ratio 


2 


RE=—  (35) 

Of 

of  the  estimated  residual  variance  of  the  data  that  has  been 
corrected  for  missing  values  to  the  residual  variance  of  the 
original  complete  data  was  the  criterion  of  interest.  The 
closer  to  1.0  the  RE  value  is,  the  better  the  estimation. 


CHAPTER  4 
RESULTS  AND  DISCUSSION 

In  this  chapter  analyses  of  the  data  for  the  four 
criteria  are  presented.  The  criteria  are  (a)  the  mean 
squared  deviations  of  regression  coefficients  for  the  data 
with  estimated  missing  values  from  the  regression 
coefficients  for  the  original  complete  data  (BSE) ;   (b)  the 
ratio  (sj/a^^)  of  the  estimated  residual  variance  of  the  data 
with  estimates  for  missing  values  to  the  residual  variance 
of  the  original  complete  data  (RE) ;   (c)  the  modified 
euclidean  norm  (D) ;  and  (d)  the  index  of  data  reconstruction 
(Q) .  For  all  the  criteria  except  RE,  smaller  values  indicate 
better  estimation.  For  the  RE  criterion,  values  closer  to 
1.0  indicate  better  estimation.  The  analyses  were  based  on 
the  data  presented  in  the  Appendix,  Table  30. 

Analysis  of  variance  was  used  to  investigate  the  effect 
on  the  criteria  of  the  following  factors:  sample  size  (N) , 
number  of  dependent  variables  (p) ,  degree  of  data  skewness 
(SK) ,  type  of  missing  data   (TM) ,  percent  of  missing  data 
(PM) ,  average  correlation  among  independent  variables  (R) , 
residual  variance  (E) ,  and  estimation  methods  (M) .  The 
significance  level  quoted  throughout  this  chapter  is  a=0.05. 
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Development  of  the  Working  Model 
There  were  five  between-subjects  factors  (N,  p,  SK,  R, 
E)   and  three  within-subjects  factors   (TM,  PM,  M) .  All 
factors  were  considered  fixed.  Initial  analyses  were 
conducted  to  determine  which  effects  to  include  in  the 
analysis  of  variance  model.  A  forward  selection  approach  was 
used,  with  all  main  effects  included  first  in  the  model, 
followed  by  all  two-way  interactions,  all  three-way 
interactions,  and  so  on.  The  statistic  used  to  compare  the 
models  was 

FSS  -  RSS 

p_   dfp  -  df^  (36) 
ESS 

where  FSS  and  ESS  are  the  model  and  error  sum  of  squares, 
respectively,  of  the  full  model,  dfp  and  dfg  are  the 
corresponding  degrees  of  freedom,  and  RSS  and  df^  are  the 
model  sum  of  squares  and  degrees  of  freedom,  respectively, 
of  the  reduced  model.  The  quantity  F  is  distributed  as  an  F- 
statistic  with  (dfp-dfR)  and  dfg  degrees  of  freedom. 

Because  there  were  between-subjects  factors  and  within- 
subjects  factors,   it  was  important  to  determine  first  which 
between-subjects  effects  to  enter  into  the  model.  Table  2 
gives  models  having  between-subjects  effects  only.  There  was 
no  significant  difference  at  a=0.05  between  the  three-factor 
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Table  2 

Magnitudes  of       for  Between-Subiects  Effects^  Models 


Variable        Highest-Order  Terms 


Q 

Main  Effects 

0.038 

Two-way  Interactions 

0.084 

Three-Way  Interactions 

0.133 

Four-Way  Interactions 

0.154 

Five-Way  Interactions 

0.158 

D 

Main  Effects 

0.143 

Two-way  Interactions 

0. 172 

Three-Way  Interactions 

0. 173 

Four-Way  Interactions 

0.174 

Five-Way  Interactions 

0.174 

RE 

Main  Effects 

0.  023 

Two-Way  Interactions 

0.059 

Three-Way  Interactions 

0.075 

Four-Way  Interactions 

0.082 

Five-Way  Interactions 

0.085 

BSE 

Main  Effects 

0.089 

Two-Way  Interactions 

0.  149 

Three-Way  Interactions 

0. 160 

Four-Way  Interactions 

0. 165 

Five-Way  Interactions 

0. 166 
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interaction  model  and  the  four-factor  interaction  model  for 
any  of  the  criteria.  Results  for  criteria  Q,  RE  and  BSE 
suggested  a  model  with  up  to  three-way  interactions,  while 
the  results  for  criterion  D  suggested  a  model  with  up  to 
only  two-way  interactions.  Thus,  the  two-way  interaction 
model  and  three-way  interaction  model  were  the  two  possible 
candidates  for  the  between-subjects  factors. 

To  determine  which  within-subjects  and  mixed  effects  to 
enter,  the  forward  selection  approach  was  used  again. 
Within-subjects  and  mixed  effects  were  entered  into  purely 
between-subjects  models.  In  Table  3,  R,^  is  the  R^  for  the 
models  with  two-factor  interactions  as  the  most  complicated 
between-subjects  effects;  R^^  is  the  R^  for  the  models  with 
three-factor  interactions  as  the  most  complicated  between- 
subjects  effects.  The  R,^  values  were  lower  than  R^^  values, 
consequently  models  relating  to  the  R^^  values  were  of 
interest.  Using  equation  36,  the  two-way  and  three-way 
interaction  models  were  not  significantly  different  at 
a=0.05;  while  the  main  effects  and  the  two-way  interaction 
models  were  significantly  different  at  a=0.5  (Table  4). 
Thus,  the  model  with  within  subjects  main  effects,  and  two- 
way  interactions  when  the  highest  order  of  between  subjects 
factors  were  the  three-way  interactions  was  selected.  Table 
5  shows  the  sources  of  variation  for  the  selected  model. 

Using  the  selected  model,  and  assuming  that  all  factors 
were  fixed,  expected  mean  square  components  were  computed 
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Table  3 

Magnitudes  of       for  Within-Subiects  Main  Effects.  Tvo-Wav 
Interaction,  and  Three-Way  Interaction  Models  when  the 
Highest-order  Between-Subiects  Factors  Model  are  the  Two- 
Way  (R.^)   or  Three-Way  (R,,^)  Interactions 


Variable  Highest-Order  Terms*  R,^  R^^ 


Q 

Main  Effects 

0.618 

0.734 

Two-way  Interactions 

0.735 

0.895 

Three-Way  Interactions 

0.746 

0.909 

D 

Main  Effects 

0 . 940 

0 . 945 

Two-way  Interactions 

0.988 

0.994 

Three-Way  Interactions 

0.991 

0.998 

RE 

Main  Effects 

0.474 

0.563 

Two-way  Interactions 

0.686 

0.867 

Three-Way  Interactions 

0.706 

0.890 

BSE 

Main  Effects 

0.499 

0.570 

Two-way  Interactions 

0.752 

0.886 

Three-Way  Interactions 

0.788 

0.936 

*  Refers  to  highest-order  terms  for  pure  within-subjects 
effects. 
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Table  4 

Model  Comparisons 


Models 

Compared  Criterion  dfy-df^  dfg  F-test 


D 

208 

174 

1.26 

RE 

208 

174 

0. 17 

BSE 

208 

174 

0.03 

Q 

104 

102 

0.16 

D 

364 

382 

9.41* 

RE 

364 

382 

2.41* 

BSE 

3  64  ' 

382 

2.91* 

Q 

208 

206 

1.51* 

*  Means  significant  at  a=0.05. 

Note.  Model  3  (2,  ME)  is  the  model  with  up  to  three-factor 
interactions  (two-factor  interactions,  main  effects) . 


Table  5 

Skeleton  Sources  of  Variation  in  the  Analysis  of  Variance 


for  the  Selected  Working  Model 


Source  of  Variation 


Between-Subjects  Factors 

-  Main  Effects  (Bl) 

-  Two-way  Interactions  (B2) 

-  Three-Way  Interactions  (B3) 
Error:  Four-Way  and  Five-Way  Interactions  (E,) 
Within-Subjects  Factors 

-  Main  Effects  (Wl) 

-  Two-way  Interactions  (W2) 


-  Bl  X  Wl 

-  Bl  X  W2 

-  B2  X  Wl 

-  B2  X  W2 

-  B3  X  Wl 

-  B3  X  W2 

Error:  Remaining  (Eb) 


for  each  effect.  Since  all  the  factors  were  assumed  to  be 
fixed,  only  the  within-subjects  error,       =  a},  and  the 
between-subjects  error,  E,  =  o}-\-x\-^y^f^viy^a^ ,  were  random  where 
^^PM  ^rid  n^  are  the  number  of  levels  for  the  factors  TM, 
PM  and  M,  respectively.  The  expected  mean  square  for  an 
effect  A  having  a  levels  is 


2   ^  j-1 

(a-1) 


where  n  is  the  product  of  the  number  of  levels  for  the 
factors  not  included  in  the  given  effect,  a„^=E,  if  the 
effect  involves  only  between-subjects  factors,  and  a„^=E,, 
otherwise.  Let 

(a-1) 

The  fixed  component  of  an  effect  can  be  estimated  as 

'*  n 

where  MSg  is  the  estimate  of  aj.  Negative  fixed  components 
were  set  to  zero.  Myers  (1979)  recommended  weighting  the 
fixed  component  by  df^/n^,  where  n^  is  the  number  of  levels 
of  the  effect,  and  df^  are  the  degrees  of  freedom  of  the 
effect.  Thus, 
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2_ 

Oa  — 

The  sum  of  the  ^^^s  plus  the  estimates  of        and  a/  give  the 
total  of  the  estimated  components  of  the  mean  square,  TMSE. 
The  proportion        to  TMSE  (W)  ,  expressed  as  a  percentage, 
indicates  the  importance  of  the  A  effect  relative  to  the 
rest.  The  proportions  for  each  criterion  will  be  reported  in 
subsequent  sections. 

Many  of  the  factors  that  would  be  expected  to  affect 
the  criteria  were  included  in  the  study.  Consequently  a 
large  number  of  significant  effects  were  expected.  However, 
not  all  of  these  would  necessarily  be  important.  Initially, 
for  each  criterion  an  effect  was  considered  important  if  (a) 
it  was  significant  and  (b)  the  W  associated  with  the  effect 
was  large  relative  to  the  W  associated  with  other  effects 
that  were  significant  for  that  criterion.  The  later  part  of 
the  criterion  for  importance  is  analogous  to  the  scree  test 
in  factor  analysis.  Inspection  of  W  for  all  four  criteria 
indicated  that  the  definition  of  an  important  effect  ranged 
from  an  effect  with  W=2.3%  to  an  effect  with  W=12.9%.  To 
compromise  between  interpreting  a  large  number  of 
unimportant  effects  and  failing  to  interpret  an  important 
effect,  the  criterion  used  to  select  effects  for 
interpretation  was  W>1.0%.  This  criterion  has  been  used  by 
Algina  and  Oshima  (1994)  and  Coombs  (1992). 
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Pairwise  comparisons  of  means  were  conducted  for 
effects  that  were  significant  and  accounted  for  at  least  1% 
of  TMSE.  The  Tukey  HSD  test  was  used  for  pairwise 
comparisons.  All  the  comparisons  were  done  at  a=0.05  level 
of  significance.  Two  means  Y;  and  ¥„  were  declared 
significantly  different  from  each  other  if  their  absolute 
difference  was  greater  than 

where  Oj  was  an  estimate  of  the  standard  error  of  the  mean 
based  on  p  degrees  of  freedom,  t  was  the  number  of  means,  a 
was  the  significance  level  set  at  0.05,  and  q(.)   is  the 
Studentized  range  critical  value.  The  test  controls  the 
familywise  error  rate  at  a.  There  were  two  standard  errors 
depending  on  whether  the  comparison  involved  one  or  two 
between-subjects  treatments.  Table  6  gives  the  formulae  for 
estimating  the  various  variances  of  the  means  and  their 
degrees  of  freedom.  Table  29  in  the  appendix  gives  the  Tukey 
critical  values  at  a=0.05  for  all  pairwise  comparisons. 

Results  for  the  Q  Criterion 
The  Q  criterion  measures  how  well  the  missing  values 
are  reconstructed.  It  is  computed  by  using  equation  (30) . 
Data  reconstruction  is  only  used  with  the  MR,  RG,   and  EM 
methods.  Reported  in  Table  7  are  effects  that  (a)  were 


Table  6 

Variances  of  the  Means  and  their  Degrees  of  Freedom 


Source  Mean  Variance  DF 

Between-Subjects  Means    EjCnT^ripMnM)"'  E,  df 

Within-Subjects  Means      Eb(nNnpnsKnRnE)''  df 
Within-Subjects  Means 
for  the  same  Between- 
Subjects  Treatment  E^  E^  df 
Between-Subjects  Means 
for  the  same  Within- 
Subjects  Treatment          [  (nTMnp^nM-l)  Eb+EJ  (nTMnp^n^)  "*  Unknown 


Table  7 

Percent  of  TMSE  for  the  Q  Criterion 

Effect  %  Fixed  Component 


M         .  13.6 

TM  12.9 

TMxM  4.0 

SKxTM  2.2 

SKxRxTM  1.3 

P  1.2 

PM     .  1.1 

pxTM  1.0 
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significant  and  (b)   accounted  for  at  least  1%  of  the  TMSE. 
Estimation  method  and  type  of  missing  data  had  the  dominant 
effects  on  Q  score.  Factors  N  and  E  had  no  apparent  effect 
on  Q  score. 

Effect  of  PM.  Four  factors  had  main  effects  that 
accounted  for  more  than  1%  of  the  TMSE.  Of  these  only  PM  did 
not  have  a  significant  interaction  with  other  factors.  The 
marginal  means  for  the  two  levels  of  PM  were  2.16  for  PM=5 
and  2.86  for  PM=15.  Data  reconstruction  was  better  when  the 
percent  missing  data  was  low.  Other  significant  main  effects 
were  for  M,  TM,  and  p.  However,  since  the  TMxM,  and  pxM  were 
significant,  the  effects  of  these  factors  were  interpreted 
from  the  two-way  tables. 

The  pxTM  effect.  A  two-way  table  of  means  is  presented 
in  Table  8.  For  censored  data,  data  reconstruction  was 
significantly  better  when  the  number  of  independent 
variables  was  high,  while  for  data  missing  at  random,  there 
were  no  significant  differences  in  data  reconstruction  as  p 
varied  from  3  to  6.  Data  reconstruction  was  best  when  data 
were  missing  at  random,  and  worst  when  data  were  right- 
censored  regardless  of  the  number  of  independent  variables. 
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Table  8 

Means  of  Q  Scores  as  a  Function  of  p  and  TM 


Type  of  Missing  Data 


Factor    Level  RM  LC  RC  p  Means 


p  3  1.003  aA       3.648  bB         4.119  cB  2.923 

6  0.860  aA       1.854  bA         3.568  cA  2.094 


TM  Means  0.931  2.751  3.843 

Note.  Within  a  row,  means  that  do  not  have  a  common  lower 
case  letter  are  significantly  different,  and  within  a 
column,  means  that  do  not  have  a  common  upper  case  letter 
are  significantly  different  at  a=0.05  using  Tukey's  HSD 
test. 


83 

The  TMxM  effect.  The  results  are  presented  in  Table  9. 
When  data  were  missing  at  random,  the  EM  and  RG  methods  were 
significantly  different  with  a  lower  Q  mean  score  for  the  EM 
method;  the  MR  method  was  not  significantly  different  from 
either  of  the  other  two  methods.  When  data  were  censored, 
the  EM  method  was  superior.  The  RG  method  outperformed  the 
MR  method  only  when  data  were  right-censored.  All  estimation 
methods  declined  in  performance  when  data  were  censored  with 
poorest  reconstruction  when  data  were  right-censored. 

The  SKxRxTM  effect.  A  three-way  table  of  means  is 
presented  in  Table  10.  For  normally  distributed  data,  data 
reconstruction  was  (a)  superior  when  data  were  missing  at 
random,  and  similar  when  data  were  left-censored  and  right- 
censored,  regardless  of  the  level  of  the  average 
correlation,  and  (b)  unaffected  for  all  three  types  of 
missing  data  as  the  average  correlation  varied  from  0.2  to 
0.7.  When  data  were  lognormally  distributed,  the  effect  the 
estimation  method  depended  on  the  average  correlation.  When 
the  average  correlation  was  0.2  data  reconstruction  was 
superior  when  data  were  either  left-censored  or  missing  at 
random;  when  the  average  correlation  was  0.7,  data 
reconstruction  was  best  with  randomly  missing  data.  Data 
reconstruction  was  (a)  unaffected  when  data  were  missing  at 
random  as  the  average  correlation  varied  from  0.2  to  0.7, 
(b)  declined  when  data  were  left-censored  as  the  average 
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Table  9 

Means  of  0  Scores  as  a  Function  of  TM  and  M 


Estimation  Method 

Factor  

Level  EM  MR  RG  TM  Means 


TM        RM  0.350  aA  1.024  abA  1.420  bA  0.931 

LC  1.269  aB  3.784  bB  3.200  bB  2.751 

RC  2.426  aC  5.539  cC  3.565  bB  3.843 


M  Means  1.348  3.449  2.728 

Note.  Within  a  row,  means  that  do  not  have  a  common  lower 
case  letter  are  significantly  different,  and  within  a 
column,  means  that  do  not  have  a  common  upper  case  letter 
are  significantly  different  at  a=0.05  using  Tukey's  HSD 
test . 
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Table  10 

Means  of  Q  Scores  as  a  Function  of  SK.  R.   and  TM. 
Factors  Type  of  Missing  data 

SK  R  RM  LC  RC 


0 

0.2 

1. 186 

aA 

3.088 

bB 

2.913 

bA 

0 

0.7 

0.707 

aA 

2  .440 

bB 

2.676 

bA 

2 

0.2 

1. 163 

aA 

1.723 

aA 

5.497 

be 

2 

0.7 

0.669 

aA 

3  .753 

bC 

4.287 

bB 

Note.  Within  a  row,  means  that  do  not  have  a  common  lower 
case  letter  are  significantly  different.  Within  a  column,  in 
comparisons  of  levels  of  one  factor  controlling  the  other 
factors,  means  that  do  not  have  a  common  upper  case  letter 
are  significantly  different  at  a=0.05. 
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correlation  varied  from  0.2  to  0.7,  and  (c)   improved  when 
data  were  right-censored  as  the  average  correlation  varied 
from  0 . 2  to  0 . 7 . 

Summary .  The  results  indicate  that  the  EM  and  MR 
methods  performed  similarly  when  data  were  randomly  missing, 
but  when  data  were  censored  the  EM  method  resulted  in  better 
reconstruction  of  missing  data  than  did  the  MR  and  RG 
methods.  The  RG  method  gave  better  data  reconstruction  than 
the  MR  method  when  data  were  right-censored,  otherwise  the 
two  estimation  methods  performed  similarly.  At  both  levels 
of  p,  reconstruction  was  best  when  data  were  missing  at 
random  and  poorest  when  data  were  right-censored.  Data 
reconstruction  improved,  though  not  significantly  in  the 
case  of  randomly  missing  data,  with  increase  in  the  number 
independent  variables.  When  data  were  normal,  reconstruction 
was  best  when  data  were  missing  at  random,  and  there  was  no 
significant  difference  between  the  two  types  of  censoring  at 
either  levels  of  the  average  correlation.  Reconstruction 
improved  slightly,  though  not  significantly,  when  the 
average  correlation  increased  from  0.2  to  0.7  for  all  types 
of  missing  data.  When  the  independent  variables  were 
lognormal,  reconstruction  was  poorest  when  data  were  right- 
censored  at  both  levels  of  the  intercorrelation. 
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Results  for  the  D  Criterion 
The  D  criterion  measures  how  well  the  variance- 
covariance  matrices  are  estimated.  It  is  computed  by  using 
equation  (31) .  Presented  in  Table  11  are  effects  that  (a) 
were  significant,  and  (b)  accounted  for  at  least  1%  of  the 
TMSE.  Type  of  missing  data  and  percent  missing  data  had  the 
dominant  effects  on  D.  The  number  of  observations  had  no 
apparent  effect  on  this  criterion. 

Effect  of  M.  Five  factors  had  main  effects  that 
accounted  for  more  than  1%  of  the  TMSE.  Of  these  only  M  did 
not  have  a  significant  interaction  with  other  factors.  The 
marginal  means  for  M  were  0.466,  0.502,  0.598,  0.629  and 
0.676  for  EM,  PP,  CD,  RG  and  MR  respectively.  Since  Tukey's 
HSD  critical  value  was  0.013,  differences  between  all  pairs 
of  means  were  significant.  The  EM  method  was  the  best  and  MR 
was  the  poorest.  The  other  significant  main  effects  were  TM, 
PM,  E,  and  R,  with  each  having  significant  interactions  with 
other  factors.  Thus,  the  effects  of  these  factors  were 
interpreted  from  the  two-way  tables. 

The  SKxTM  effect.  A  two-way  table  of  means  is  presented 
in  the  top  panel  of  Table  12.  The  estimation  of  the  V-C 
matrices  declined  when  data  were  censored  for  both  normal 
and  lognormal  data.  However,  the  estimation  was  better  when 
data  were  left-censored  than  right-censored  data  only  when 


Table  11 

Percent  of  TMSE  for  the  D  Criterion 


Effect  %  Fixed  Component 


TM 

44 . 9 

PM 

19.1 

E 

10.2 

M 

4.0 

ExTM 

2.1 

TMxPM 

2.1 

R 

1.6 

pxR 

1.4 

PMxE 

1.2 

SKxTM 

1.1 
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Table  12 

Means  of  D  Scores  as  a  Function  of  E  and  TM.  TM  and  SK.  and 


TM  and  PM. 


Factor  Level 


Type  of  Missing  Data 


RM 


LC 


RC 


Overall 
Means 


SK  0 
2 


0.231  aB  0.784  bB  0.795  bA  0.603 
0.187  aA       0.623  bA     0.828  cB  0.546 


E  6 
12 


0.163  aA  0.533  bA  0.651  cA  0.449 
0.254  aB       0.873  bB     0.971  cB  0.700 


PM  5 
15 


0.117  aA  0.500  bA  0.593  cA  0.403 
0.301  aB       0.907  bB     1.030  cB  0.746 


TM  Means 


0.209 


0.703 


0.811 


Note.  Within  a  row,  means  that  do  not  have  a  common  lower 
case  letter  are  significantly  different,  and  within  a  column 
and  panel,  means  that  do  not  have  a  common  upper  case  letter 
are  significantly  different  at  a=0.05  using  Tukey's  HSD 
test. 
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data  were  lognormal.  The  estimation  was  better  for  lognormal 
data  than  normal  data  when  data  were  missing  at  random  or 
left-censored,  but  the  opposite  was  the  case  when  data  were 
right-censored. 

The  TMxPM  and  ExTM  effects.  Two-way  tables  of  means  are 
presented  in  the  lower  two  panels  of  Table  12 .  Estimation  of 
the  V-C  matrices  was  best  when  (a)  data  were  missing  at 
random  regardless  of  the  level  of  E  or  PM,   (b)  percent 
missing  data  was  low  for  all  types  of  missing  data,  and  (c) 
the  conditional  variance  was  small  for  all  types  of  missing 
data. 

The  ExPM  effect.  A  two-way  table  of  means  is  presented 
in  Table  .13.  Estimation  of  the  V-C  matrices  was  best  when 
the  percent  of  missing  data  was  small,  for  both  levels  of 
the  conditional  variance.  For  both  percent  missing  data, 
estimation  was  best  when  the  conditional  variance  was 
smaller. 

The  pxR  effect.  A  two-way  table  of  means  is  presented 
in  Table  14.  At  low  average  correlation,  the  estimation  of 
the  V-C  matrices  was  best  when  the  number  of  independent 
variables  was  high.  At  high  average  correlation,  the 
estimation  was  unaffected  as  p  increased  from  3  to  6.  When  p 
was  low,  an  increase  in  the  average  correlation  did  not 
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Table  13 

Means  of  D  Scores  as  a  Function  of  PM  and  TM. 


Percent  Missing  Data 


Factor  Level  5  15  E  Means 


E  6  0.320  aA  0.578  bA  0.449 

12  0.486  aB  0.913  bB  0.700 


PM  Means  0.403  0.746 


Note.  Within  a  row,  means  that  do  not  have  a  common  lower 
case  letter  are  significantly  different,  and  within  a 
column,  means  that  do  not  have  a  common  upper  case  letter 
are  significantly  different  at  a=0.05  using  Tukey's  HSD 
test. 
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Table  14 

Means  of  D  Scores  as  a  Function  of  p  and  R. 


Number  of  Independent  Variables 


Factor    Level  3  6  R  Means 


R  .  0.2  0.687  bA  0.490  aA  0.392 

0.7  0.695  aA  0.699  aB  0.465 


p  Means  0.691  0.594 

Note.  Within  a  row,  means  that  do  not  have  a  common  lower 
case  letter  are  significantly  different,  and  within  a 
column,  means  that  do  not  have  a  common  upper  case  letter 
are  significantly  different  at  a=0.05  using  Tukey's  HSD 
test. 
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significantly  affect  the  estimation,  but  when  p  was  high, 
the  estimation  accuracy  was  significantly  reduced  with 
increase  in  the  average  correlation. 

Summary .  The  results  indicate  that  the  EM  method 
resulted  in  better  estimation  of  the  V-C  matrices  than  did 
the  other  estimation  methods.  Estimation  was  better  when  the 
conditional  variance,  or  percent  missing  data  was  small  for 
all  types  of  missing  data.  When  the  average  correlation 
was  low,  estimation  improved  when  the  number  of  independent 
variables  was  high,  but  when  the  average  correlation  was 
high,  the  estimation  was  unaffected  as  p  increased  from  3  to 
6.  Estimation  was  best  when  data  were  missing  at  random  for 
(a)  normal  and  lognormal  data,    (b)   small  and  large 
conditional  variances,  and  (c)  small  and  large  percent 
missing  data.  Estimation  was  better  for  left-censored  data 
than  right-censored  data  regardless  of  the  levels  of  E  or 
PM,  and  for  lognormal  data.  Of  course,  because  of  the 
symmetry  of  data  distribution  when  data  were  normal, 
estimation  was  similar  for  left-censored  data  and  right- 
censored  data. 

Results  for  the  RE  Criterion 
The  RE  criterion  measures  how  well  the  conditional 
variance  is  estimated.  It  is  computed  by  using  equation 
(35) .  The  closer  the  RE  scores  are  to  1,  the  better  the 
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estimation.  Presented  in  Table  15  are  effects  that  (a)  were 
significant,  and  (b)  accounted  for  at  least  1%  of  the  TMSE. 
Percent  missing,  type  of  missing  data,  and  estimation  method 
had  dominant  effects  on  RE,  however  none  were  outstanding. 

Three  factors  had  main  effects  that  accounted  for  at 
least  1%  of  TMSE.  All  of  these  had  significant  interactions 
with  other  factors;  consequently  the  effects  of  these 
factors  were  interpreted  from  tables  of  interaction  effects. 

The  NxExTMxM  effect.  A  four-way  table  of  means  is 
presented  in  Table  16.  Tukey's  HSD  test  was  conducted  for 
all  factors  but  Table  16  reports  only  for  the  comparison  of 
the  five  estimation  methods.  In  general  the  differences 
between  the  methods  at  each  combination  of  N,  E,  and  TM  were 
small,  though  a  few  significant  differences  were  detected. 
The  N  effects,  holding  E,  TM,  and  M  constant  were  generally 
small.  Similarly,  the  E,  TM  and  M  effects  were  small. 
Generally,  for  each  combination  of  N  and  E,  estimation 
accuracy  of  each  method  tended  to  decline,  though  not 
significantly,  when  data  were  censored,  with  right-censored 
data  most  affected. 

The  pxRxM  effect.  A  three-way  table  of  means  is 
presented  in  Table  17.  The  results  showed  that  (a)  when  p=3 , 
all  the  estimation  methods  performed  similarly  regardless  of 
the  level  of  the  average  correlation,    (b)  when  p=6  and 
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Table  15 

Percent  of  TMSE  for  Criterion  RE 


Effect  %  Fixed  Component 


PM  7.4 

TM  4.8 

M  -  3.1 

SKxM  2.3 

SKxTM  1.8 

pxM  1.7 

NxTMxM  1.5 

NxExTMxM  1.5 

pxRxM  1.4 

TMxM  1.3 

pxTM  1.2 

PMxM  1.2 

NxPM  1 . 1 
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Table  16 

Means  of  RE  Scores  as  a  Function  of  N.   E,  TM.  and  M. 


Factors  Estimation  Methods 


N  E        TM        CD  EM  MR  PP  RG 


100 

6 

RM 

1 . 

035 

ab 

0. 

972 

a 

0. 

976 

a 

1. 

058 

b 

1. 

048 

b 

100 

6 

LC 

1. 

027 

a 

0. 

993 

a 

1. 

007 

a 

1. 

007 

a 

1. 

032 

a 

100 

6 

RC 

1. 

117 

b 

1. 

046 

a 

1. 

034 

a 

1. 

043 

a 

1. 

216 

c 

100 

12 

RM 

1. 

034 

ab 

0. 

967 

a 

0. 

969 

a 

1. 

047 

b 

1. 

026 

ab 

100 

12 

LC 

1. 

038 

a 

0. 

994 

a 

1. 

003 

a 

1. 

009 

a 

1. 

007 

a 

100 

12 

RC 

1. 

102 

be 

1. 

025 

a 

1. 

027 

a 

1. 

035 

ab 

1. 

126 

c 

300 

6 

RM 

1. 

008 

a 

0. 

992 

a 

0. 

996 

a 

1. 

026 

a 

1. 

033 

a 

300 

6 

LC 

1. 

031 

a 

1. 

009 

a 

1. 

017 

a 

1. 

013 

a 

1. 

006 

a 

300 

6 

RC 

1. 

050 

a 

1. 

014 

a 

1. 

020 

a 

1. 

271 

c 

1. 

099 

b 

300 

12 

RM 

1. 

012 

a 

0. 

996 

a 

0. 

997 

a 

1. 

005 

a 

1. 

030 

a 

300 

12 

LC 

1. 

043 

a 

1. 

020 

a 

1. 

024 

a 

1. 

020 

a 

1. 

012 

a 

300 

12 

RC 

1. 

062 

a 

1. 

024 

a 

1. 

027 

a 

1. 

027 

a 

1. 

080 

a 

M  Means 

1. 

047 

1. 

004 

1. 

008 

1. 

047 

1. 

060 

Note.  Within  a  row,  means  that  do  not  show  a  common  letter 
are  significantly  different  at  a=0.05  using  Tukey's  HSD 
test.  Within  a  column,  when  N  and  E  are  fixed  the  critical 
value  for  comparing  TM  means  is  0.146,  when  N  and  TM  are 
fixed  the  critical  value  for  comparing  E  means  is  0.122,  and 
when  E  and  TM  are  fixed  the  critical  value  for  comparing  N 
means  is  0.122. 
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Table  17 

Means  of  RE  Scores  as  a  Function  of  p.  R.   and  M. 


Factors  Estimation  Methods 


p         R  CD  .    EM  MR  PP  RG 


3 

0.2 

1. 

043 

aA 

1. 

014 

aA 

1. 

017 

aA 

1. 

016 

aA 

1. 

018 

aA 

3 

0.7 

1. 

031 

aA 

1. 

003 

aA 

1. 

004 

a  A 

1. 

025 

aA 

1. 

021 

aA 

6 

0.2 

.  1. 

072 

bB 

1. 

003 

aA 

1. 

008 

aA 

1. 

Oil 

aA 

1. 

033 

aA 

6 

0.7 

1. 

041 

bA 

0. 

998 

aA 

1. 

003 

abA 

1. 

135 

cB 

1. 

166 

cB 

M 

Means 

1. 

047 

1. 

004 

1. 

008 

1. 

047 

1. 

060 

Note.  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different.  Within  a  column,   in  comparisons 
of  levels  of  one  factor  controlling  the  other  factors,  means 
that  do  not  have  a  common  upper  case  letter  are 
significantly  different  at  a=0.05. 
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R=0.2,  all  the  estimation  methods  performed  similarly  except 
for  the  CD  method  which  performed  more  poorly  than  the  rest, 
and  (c)  when  p=6  and  R=0.7,  the  EM  and  MR  methods  performed 
similarly  and  were  the  best,  while  the  PP  and  RG  methods 
were  the  poorest.  The  p  and  R  factors  affected  the 
estimation  accuracy  only  for  the  CD,  PP  and  RG  methods.  With 
p=6,  accuracy  of  the  PP  and  RG  methods  declined  while  that 
of  the  CD  method  improved,  as  the  correlation  increased. 

The  SKxTM.  and  pxTM  effects.  Two-way  tables  for 
interpreting  the  SKxTM  and  pxTM  effects  are  reported  in 
Table  18.  The  estimation  of  the  conditional  variance  was 
similar  for  missing  at  random  data  and  left-censored  data 
regardless  of  the  degree  of  skewness  or  the  number  of 
independent  variables.  When  data  were  left-censored, 
estimation  was  better  for  lognormal  data  than  for  normal 
data.  The  opposite  was  the  case  when  data  were  right- 
censored.  As  p  increased  from  3  to  6,  estimation  from  the 
randomly  missing  data  and  the  right-censored  data  became 
poorer  while  that  from  the  left-censored  data  was 
unaffected. 

The  NxPM  effect.  A  two-way  table  of  means  is  presented 
in  Table  19.  The  estimation  of  the  conditional  variance  was 
significantly  poorer  as  the  percent  missing  data  increased 
from  5%  to  15%  regardless  of  the  sample  size.  When  the 
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Table  18 

Means  of  RE  Scores  as  a  Function  of  SK  and  TM.   and  p  and  TM. 

Type  of  Missing  Data 
 Overall 

Factor    Level        RM  LC  RC  Means 


SK  0  1.012  aA       1.028  abB     1.044  bA  .  1.028 

2  1.011  aA       1.004  aA       1.101  bB  1.039 


p  3  0.999  aA       1.018  aA       1.041  bA  1.353 

6  1.024  aB       1.013  aA       1.104  bB  1.047 


TM  Means  1.011  1.016  1.072 

Note .  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different,  and  within  a  column  and  panel, 
means  that  do  not  have  a  common  upper  case  letter  are 
significantly  different  at  a=0.05  using  Tukey's  HSD  test. 
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Table  19 

Means  of  RE  Scores  as  a  Function  of  N  and  PM 


Percent  Missing  Data 


Factor  Level 


15 


N  Means 


N 


100 
300 


0.986  aA 
1.011  aA 


1.082  bB 
1.053  bA 


1.034 
1.032 


PM  Means 


0.999 


1.067 


Note.  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different,  and  within  a  column,  means  that 
do  not  have  a  common  upper  case  letter  are  significantly 
different  at  a=0.05  using  Tukey's  HSD  test. 
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percent  missing  data  was  low,  the  estimation  slightly 
improved  but  not  significant  at  a=0.05,  as  sample  size 
increased.  When  the  percent  missing  data  was  high,  the 
estimation  significantly  improved  as  sample  size  increased. 

The  SKxM.  and  PMxM  effects.  Two-way  tables  of  means  are 
presented  in  Table  20.  From  the  SKxM  table  of  means,  the 
results  showed  that  (a)   for  normally  distributed  data,  the 
EM,  MR  and  PP  methods  performed  similarly  and  were  the  best, 
(b)   for  lognormally  distributed  data,  the  EM  and  MR 
performed  similarly  and  were  the  best,  and  (c)  only  the  PP 
method  was  affected  by  the  skewness  of  the  data  with  the 
performance  declining  as  skewness  increased  from  0  to  2. 

From  the  PMxM  table  of  means,  the  results  showed  that 
(a)  when  the  percent  missing  data  was  low,  all  the 
estimation  methods  performed  similarly,    (b)  when  the  percent 
missing  data  was  high,  the  EM  and  MR  performed  similarly  and 
were  the  best,  while  the  RG  method  was  the  worst,  and  (c) 
the  performance  of  all  the  estimation  methods  declined  as 
percent  missing  data  increased  from  5%  to  15%. 

Summary .  The  results  indicate  that  (a)  when  data  were 
normally  distributed,  the  EM,  MR  and  PP  methods  performed 
similarly  and  were  the  best,  and  when  data  were  lognormally 
distributed,  the  EM  and  MR  methods  were  the  best,    (b)  when 
percent  missing  data  was  low,  all  the  estimation  methods 
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Table  20 

Means  of  RE  Scores  as  a  Function  of  SK  and  M.   and  PM  and  M 


Estimation  Methods 

Factor   Overall 

Level        CD  EM  MR  PP  RG  Means 


SK     0       1.041  bA     liOOO  aA     1.010  aA  1.020  aA  1.067  bA  1.028 
2       1.052  bA     1.009  aA     1.006  aA  1.073  bB  1.052  bA  1.038 


PM     5       0.996  aA     0.986  aA     0.989  aA  1.014  aA  1.009  aA  0.999 
15       1.098  bcB  1.023  aB     1.027  aB  1.079  bB  1.111  cB  1.068 


M  Means  1.047  1.004  1.008         1.047  1.060 

Note.  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different,  and  within  a  column  and  panel, 
means  that  do  not  have  a  common  upper  case  letter  are 
significantly  different  at  a=0.05  using  Tukey's  HSD  test. 
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performed  similarly,  but  when  percent  missing  data  was  high, 
the  EM  and  MR  performed  similarly  and  were  the  best,    (c)  the 
performance  of  all  the  estimation  methods  declined  as  the 
percent  missing  data  increased  from  5%  to  15%,   and  (d) 
increase  in  skewness  negatively  affected  the  PP  method  only. 
The  estimation  improved  as  the  sample  size  increased  from 
100  to  3  00  when  the  percent  missing  data  was  low,  while  the 
opposite  was  the  case  when  the  percent  missing  data  was 
high.  The  results  showed  that  the  performance  of  the  PP  and 
RG  methods  were  negatively  affected  by  increase  in  both  p 
and  R,  while  the  performance  of  the  EM,  MR  and  CD  methods 
were  unaffected. 

Results  for  the  BSE  Criterion 
The  BSE  criterion  measures  the  mean  squared  deviations 
of  regression  coefficients  for  the  data  with  estimated 
missing  values  from  the  regression  coefficients  for  the 
original  complete  data.  It  is  computed  by  using  equation 
(33) .  Smaller  values  indicate  better  estimation  method. 
Reported  in  Table  21  are  effects  that  (a)  were  significant 
and  (b)   accounted  for  at  least  1%  of  the  TMSE.  The 
significant  main  effects  were  for  R,  M,  p,   and  PM.  However, 
since  these  factors  had  significant  interactions  with  other 
factors,  the  effects  of  these  factors  will  be  interpreted 
from  tables  of  interaction  effects.  No  effects  involving  N 
or  E  satisfying  the  above  criteria  were  found. 


Table  21 

Percent  of  TMSE  for  the  BSE  Criterion 


Effect  %  Fixed  Component 


pXPMXM 

R  q 

o  •  ^ 

R 

'it/ 

pxR 

A  1 

RXM 

"  TO 
O.J 

pXM 

H 

3.5 

pxRxM 

2.8 

P 

2.7 

TMxM 

2.3 

PM 

2.0 

RxPMxM 

1.8 

PMxM 

1.7 

RXTMXM 

1.7 

SKxTMxM 

1.1 
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The  pxPMxM  effect.  A  three-way  table  of  means  is 
presented  in  Table  22.  When  p=3,  all  the  estimation  methods 
performed  similarly  regardless  of  the  level  of  PM,  except 
the  CD  method  which  was  inferior  to  the  rest  when  percent 
missing  data  was  high.  When  p=6,  the  EM,  MR  and  CD  methods 
performed  similarly  and  were  the  best  regardless  of  the 
percent  missing  data.  Fixing  the  percent  missing  data,  the 
results  showed  that  as  the  number  of  independent  variables 
increased  from  3  to  6,  the  performance  of  (a)  the  EM,  MR  and 
CD  was  unaffected,  and  (b)  the  PP  and  RG  methods  declined. 

When  the  number  of  independent  variables  are  fixed,  the 
results  showed  that  as  the  percent  missing  data  increased, 
the  performance  of  (a)  the  CD  method  declined  regardless  of 
the  number  of  independent  variables,    (b)  the  PP  and  RG 
methods  declined  only  when  p=3  and  p=6  respectively,  and  (c) 
the  EM  and  MR  methods  declined,  though  the  difference  was 
not  statistically  significant. 

The  pxRxM  effect.  A  three-way  table  of  means  is 
presented  in  Table  23.  When  p=6  and  R=0.7,  the  EM,  MR,  and 
CD  methods  performed  similarly  and  better  than  the  PP  and  RG 
methods.  For  other  combinations  of  p  and  R,  all  methods 
performed  similarly. 

When  the  intercorrelation  increased  from  0.2  to  0.7, 
the  performance  of  all  the  estimation  methods  was  (a) 
unaffected  when  p=3,  and  (b)  reduced  when  p=6.  When  the 
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Table  22 

Means  of  BSE  Scores  as  a  Function  of  p.  PM.  and  M 


Factors  Estimation  Methods 


p 

PM 

CD 

EM 

MR 

PP 

RG 

3 

5 

0. 

025 

aA 

0. 

013 

aA 

0. 

019 

aA 

0. 

016 

aA 

0.  021 

aA 

3 

15 

0. 

150 

bB 

0. 

055 

aA 

0. 

088 

aAB 

0. 

100 

aB 

0.078 

aA 

6 

5 

0. 

067 

aA 

0. 

049 

aA 

0. 

052 

aAB 

0. 

228 

be 

0.166 

bB 

6 

15 

0. 

131 

abB 

0. 

082 

aA 

0- 

092 

aB 

0. 

201 

bC 

0.665 

CC 

M 

Means 

0. 

093 

0. 

050 

0. 

063 

0. 

136 

0.227 

Note.  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different.  Within  a  column,  in  comparisons 
of  levels  of  one  factor  controlling  the  other  factors,  means 
that  do  not  have  a  common  upper  case  letter  are 
significantly  different  at  a=0.05. 
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Table  23 

Means  of  BSE  Scores  as  a  Function  of  p.  R.  and  M 


Factors 

Estimation 

Methods 

p  R 

CD 

EM 

MR 

PP 

RG 

3  0.2 

0.089 

aAB 

0.  039 

aA 

0.062 

aAB 

0.  045 

aA 

0.031 

aA 

3  0.7 

0.086 

aA 

0.029 

aA 

0.045 

aA 

0.071 

aA 

0.068 

aA 

6  0.2 

0.040 

aA 

0.  013 

aA 

0.022 

aA 

0.015 

aA 

0.035 

aA 

6  0.7 

0.159 

aB 

0.118 

aB 

0.123 

aB 

0.413 

bB 

0.796 

cB 

Note.  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different.  Within  a  column,  in  comparisons 
of  levels  of  one  factor  controlling  the  other  factors, means 
that  do  not  have  a  common  upper  case  letter  are 
significantly  different  at  a=0.05. 
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number  of  independent  variables  increased  from  3  to  6,  the 
performance  of  all  the  estimatiom  methods  was   (a)  unaffected 
when  R=0.2,  and  (b)  reduced  when  R=0.7. 

The  RxPMxM  effect.  A  three-way  table  of  means  is 
reported  in  Table  24.  When  R=0.2,  all  the  estimation  methods 
performed  similarly  regardless  of  the  level  of  PM.  When 
R=0.7,  the  EM,  MR  and  CD  methods  performed  similarly  and 
were  the  best  regardless  of  the  level  of  PM.  As  the  percent 
missing  data  increased  from  5%  to  15%,  only  the  CD  method 
declined  in  performance.  The  EM,  MR  and  PP  methods  were 
unaffected  regardless  of  the  average  correlation. 
Performance  of  the  RG  method  declined  only  when  the  average 
correlation  was  high.  Similarly,  with  percent  missing  data 
fixed,  as  the  average  correlation  increased,  the  performance 
of  (a)  the  PP  and  RG  methods  declined  regardless  of  the 
percent  missing  data,   (b)  the  CD  method  declined  only  when 
the  percent  missing  data  was  high,  and  (c)  the  EM  and  MR 
methods  were  unaffected. 

The  RxTMxM  effect.  A  three-way  table  of  means  is 
presented  in  Table  25.  When  R=0.2,  all  the  estimation 
methods  performed  similarly  regardless  of  the  type  of 
missing  data.  When  R=0.7,   similar  results  were  observed 
except  that  the  RG  method  was  inferior  to  the  rest,  and  the 
PP  method  performed  poorly  when  data  were  missing  at  random. 
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Table  24 

Means  of  BSE  Scores  as  a  Function  of  R.  PM,   and  M. 


Factors  Estimation  Methods 


R        PM        CD  EM  MR  PP  RG 


0.2 

5 

0. 

020 

aA 

0. 

009 

aA 

0. 

014 

aA 

0. 

Oil 

aA 

0. 

013 

aA 

0.2 

15 

0. 

108 

aB 

0. 

042 

aAB 

0. 

070 

aAB 

0. 

049 

aA 

0. 

053 

aA 

0.7 

5 

0. 

072 

aAB 

0. 

053 

aAB 

0. 

057 

aAB 

0. 

233 

bB 

0. 

174 

bB 

0.7 

15 

0. 

173 

aC 

0. 

095 

aB 

0. 

110 

aB 

0. 

252 

bB 

0. 

691 

CC 

Note.  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different.  Within  a  column,  in  comparisons 
of  levels  of  one  factor  controlling  the  other  factors,  means 
that  do  not  have  a  common  upper  case  letter  are 
significantly  different  at  a=0.05. 
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Table  25 

Means  of  BSE  Scores  as  a  Function  of  R.  TM.  and  M. 


Factors  Estimation  Methods 


R 

TM 

CD 

EM 

MR 

PP 

R6 

0.2 

RM 

0.005 

a 

0.002 

a 

0.007 

a 

0.002 

a 

0.026 

a 

0.2 

LC 

0.090 

a 

0.043 

a 

0.065 

a 

0.  047 

a 

0.011 

a 

0.2 

RC 

0.098 

a 

0.032 

a 

0.054 

a 

0.040 

a 

0.062 

a 

0.7 

RM 

0.080 

a 

0.073 

a 

0.040 

a 

0.552 

b 

0.494 

b 

0.7 

LC 

0.099 

a 

0.055 

a 

0. 120 

a 

0.  086 

a 

0.  278 

b 

0.7 

RC 

0.188 

a 

0.093 

a 

0.091 

a 

0.  089 

a 

0.525 

b 

Note.  Within  a  row,  means  that  do  not  show  a  common  letter 
are  significantly  different.  Within  a  column,  in  comparisons 
of  levels  of  one  factor  controlling  the  other  factors,  means 
that  di  not  have  a  common  upper  case  letter  are 
significantly  different  at  a=0.05. 


Ill 

The  PP  method,  especially  when  data  were  missing  at  random, 
and  the  RG  method  were  heavily  affected  by  an  increase  in 
average  correlation  regardless  of  the  type  of  missing  data. 

The  SKxTMxM  effect.  A  three-way  table  of  means  is 
presented  in  Table  26.  When  the  data  were  normal,  all  the 
estimation  methods  except  the  RG  method  performed  similarly 
regardless  of  the  type  of  missing  data.  When  the  data  were 
lognormal,  with  the  following  exceptions  the  estimation 
methods  performed  similarly  regardless  of  the  type  of 
missing  data:    (a)  when  data  were  randomly  missing,  then  the 
performance  of  the  PP  and  RG  methods  was  significantly 
poorer,  and  (b)  when  data  were  right-censored,  then  the 
performance  of  the  RG  method  was  significantly  poorer.  The 
performance  of  all  the  estimation  methods,  except  the  PP  and 
RG  methods,  declined  though  not  significantly,  when  data 
were  censored. 

Summary .  The  results  of  the  BSE  criterion  showed  that 
as  the  percent  missing  data  increased  from  5%  to  15%,  the 
performance  of  (a)  the  CD  method  declined  regardless  of  the 
level  of  p,    (b)  the  PP  method  declined  only  when  p=3,  (c) 
the  RG  method  declined  only  when  p=6,  and  (d)   the  EM  and  MR 
methods  slightly  declined,  though  not  significantly  at 
a=0.05,  regardless  of  the  level  of  p.  Furthermore,  as  the 
number  of  independent  variables  increased  from  3  to  6,  the 
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Table  2  6 

Means  of  BSE  Scores  as  a  Function  of  SK.  TM.   and  M 


Factors  Estimation  Methods 


SK 

TM 

CD 

EM 

MR 

PP 

RG 

0 

KM 

0.046 

a 

0.041 

a 

0.017 

a 

0.083 

a 

0.219 

b 

0 

LC 

0.098 

a 

0.037 

a 

0.067 

a 

0.057 

a 

0.234 

b 

0 

RC 

0.112 

a 

0.053 

a 

0.067 

a 

0.054 

a 

0.365 

b 

2 

BM 

0.040 

a 

0.033 

a 

0.030 

a 

0.472 

c 

0.300 

b 

2 

LC 

0.090 

a 

0.062 

a 

0. 117 

a 

0.  077 

a 

0.  056 

a 

2 

RC 

0.175 

ab 

0.072 

a 

0.079 

a 

0.075 

a 

0.221 

b 

Note.  Within  a  row,  means  that  do  not  have  a  common  letter 
are  significantly  different  at  a=0.05  using  Tukey's  HSD 
test.  Within  a  column,  when  SK  is  fixed  the  critical  value 
for  comparing  TM  means  is  0.103,   and  when  TM  is  fixed  the 
critical  value  for  comparing  SK  means  is  0.086. 


performance  of  (a)  the  EM,  MR  and  CD  was  unaffected,  and  (b) 
the  PP  and  RG  methods  declined  for  all  levels  of  PM. 

As  the  number  of  independent  variables  increased  from  3 
to  6,  the  performance  of  all  the  estimation  methods  was  (a) 
unaffected  when  the  average  correlation  was  low,  and  (b) 
declined  when  the  average  correlation  was  high. 

Summary  with  Respect  to  the  Estimation  Methods 

For  estimating  V-C  matrices,  there  were  no  interactions 
with  estimation  methods.  However,  comparisons  among  the 
estimation  methods  showed  that  the  EM  method  was  the  most 
accurate.  For  the  other  criteria — reconstruction  of  missing 
values  (Q) ,  estimation  of  the  conditional  variance  (RE) ,  and 
estimation  of  the  regression  coefficients  (BSE) — there  were 
significant  interactions  with  estimation  methods.  For  each 
criterion  and  each  significant  interaction  with  estimation 
methods,  the  most  accurate  estimation  methods  are  displayed 
in  Table  27. 
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Table  27 

The  Best  Estiiaation  Methods  as  Other  Factors  are  Varied 


Criteria 

Factors   


and  Levels      Q  RE  BSE 


TM     RM  EM  MR 

LC  EM 
RC  EM 


SK 

0 

EM 

MR 

PP 

2 

EM 

MR 

PM 

5 

CD 

EM 

MR 

PP 

RG 

15 

EM 

MR 

P 

R 

3 

0.2 

CD 

EM 

MR 

PP 

RG 

CD 

EM 

MR 

PP 

RG 

3 

0.7 

CD 

EM 

MR 

PP 

RG 

CD 

EM 

MR 

PP 

RG 

6 

0.2 

EM 

MR 

PP 

RG 

CD 

EM 

MR 

PP 

RG 

6 

0.7 

EM 

MR 

CD 

EM 

MR 
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Table  27  —  Continued. 


Factors  Criteria 

and  Levels   

p      PM  BSE 


3 

5 

CD 

EM 

MR 

PP 

RG 

3 

15 

EM 

MR 

PP 

RG 

6 

5 

CD 

EM 

MR 

6 

15 

CD 

EM 

MR 

R 

PM 

0.2 

5 

CD 

EM 

MR 

PP 

RG 

0.2 

15 

CD 

EM 

MR 

PP 

RG 

0.7 

5 

CD 

EM 

MR 

0.7 

15 

CD 

EM 

MR 

R 

TM 

0.2 

RM 

CD 

EM 

MR 

PP 

RG 

0.2 

LC 

CD 

EM 

MR 

PP 

RG 

0.2 

RC 

CD 

EM 

MR 

PP 

RG 

0.7 

RM 

CD 

EM 

MR 

0.7 

LC 

CD 

EM 

MR 

PP 

0.7 

RC 

CD 

EM 

MR 

PP 

Table  27  —  Continued. 


Factors   

and  Levels  BSE 


Criteria 


SK  TM 

0  RM  CD  EM  MR  PP 

0  LC  CD  EM  MR  PP 

0  RC  CD  EM  MR  PP 

2  RM  CD  EM  MR 

2  LC  CD  EM  MR  PP  RG 

2  RC  CD  EM  MR  PP 


Table  27  —  Continued. 


Criteria 

Factors   

and  Levels  RE 


N        E  TM 

100     6  RM 

100     6  LC 

100     6  RC 

100  12  RM 

100  12  LC 

100  12  RC 

300     6  RM 

300     6  LC 

300     6  RC 

300  12  RM 

300  12  LC 

300  12  RC 


CD  EM 
CD  EM 

EM 
CD  EM 
CD  EM 

EM 
CD  EM 
CD  EM 
CD  EM 
CD  EM 
CD  EM 
CD  EM 


MR 

MR  PP  RG 
MR  PP 
MR  RG 
MR  PP  RG 
MR  PP 
MR  PP  RG 
MR  PP  RG 
MR 

MR  PP  RG 
MR  PP  RG 
MR  PP  RG 


CHAPTER  5 
CONCLUSIONS 

One  hundred  ninety-two  simulated  conditions  were  used 
to  compare  the  effects  on  four  measures  of  performance  of 
five  estimation  methods.  Although  three  methods,  namely  CD, 
MR  and  PP,  are  ad  hoc  in  character,  they  are  commonly  used 
in  missing  value  problems.  On  the  other  hand,  the  EM 
algorithm  and  the  RG  method  are  fortified  with  statistical 
arguement  and  derivation.  The  use  of  the  EM  method  is  often 
avoided  because  of  its  complexity  and  demanding  computing 
skills. 

The  four  measures  of  performance  were  (a)  the  mean  sum 
of  squares  of  deviations  of  regression  coefficients  of  the 
data  that  has  been  corrected  for  missing  values  from  the 
regression  coefficients  of  the  original  complete  data,  BSE; 
(b)  the  ratio  {s^/a^)   of  the  estimated  residual  variance  of 
the  data  that  has  been  corrected  for  missing  values  to  the 
residual  variance  of  the  original  complete  data,  RE;   (c)  the 
modified  euclidean  norm,  D;  and  (d)   the  index  of  data 
reconstruction,  Q.  Missing  data  are  not  reconstructed  when 
the  CD  and  PP  methods  are  used,  consequently  the  CD  and  PP 
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methods  were  not  involved  when  the  Q  criterion  results  were 
considered. 

The  results  obtained  may  be  applied  to  experiments 
which  have  experimental  conditions  similar  to  the  studied 
192  conditions.  Consequently,  the  generalizability  of  the 
results  is  limited  by  the  range  of  values  for  the  degree  of 
data  skewness  (SK) ,  average  intercorrelation  of  variables 
(R) ,  percent  of  missing  entries  (PM) ,  type  of  missing  data 
(TM) ,  residual  variance  (E) ,  sample  size  (N) ,  and  number  of 
independent  variables  (P) .  Within  these  limitations,  the 
following  findings  of  the  present  study  and  comparisons  to 
previous  literature  can  be  made. 

Based  on  the  Estimation  of  Missing  Values 
Past  research  has  shown  that 

1.  The  MR  method  was  better  than  the  RG  method  when  the 
average  correlation  of  the  independent  variables  was  less 
than  0.2,  otherwise  the  RG  method  was  superior  (Gleason  & 
Staelin,   1975) . 

2.  When  the  percent  missing  data  was  small   (PM=5) ,  the 
EM,  CD  and  PP  methods  performed  similarly,  except  when 
correlational  structure  was  high  (R=0.75,  R^=0.9),  then  the 
PP  method  was  no  longer  competitive  (Azen  et  al.,  1989). 

3.  When  percent  missing  data  was  large  (PM=25) ,  the  EM 
method  was  better  than  the  CD  and  PP  methods  (Azen  et  al., 
1989). 
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4.  Data  reconstruction  was  poorer  when  data  were 
lognormal  or  censored  (Azen  et  al.,  1989). 

The  results  of  the  present  study  have  shown  that: 

1.  Data  reconstruction  was  better  when  the  percent 
missing  data  was  low. 

2 .  Reconstruction  was  better  when  data  were  missing  at 
random  than  censored  data  regardless  of  the  degree  of 
skewness,  average  correlation,  or  number  of  independent 
variables. 

3 .  The  EM  method  gave  the  best  data  reconstruction  for 
all  types  of  missing  data,  and  the  MR  method  performed 
similarly  as  (a)  the  EM  method  only  when  data  were  missing 
at  random,  and  (b)  the  RG  method  except  when  data  were 
right-censored  then  the  MR  method  was  the  poorest. 

Thus,  the  present  study  (a)  verifies  the  superiority  of 
the  EM  method,  and  (b)   shows  that  among  the  ad  hoc  methods 
the  MR  method  is  as  well  as  the  EM  method  when  data  are 
randomly  missing,    (c)   shows  that  when  data  are  censored,  the 
MR  is  no  longer  competitive,  and  (d)   shows  that  the 
performance  of  all  the  estimation  methods  declines  when  data 
are  censored. 

Based  on  the  Estimation  of  Regression  Coefficients 
This  has  been  the  most  commonly  used  criterion  for 

assessing  the  performance  of  the  estimation  methods.  Past 

research  has  shown  that 
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1.  Based  on  the  analysis  of  pivotal  (equation  29),  the 
EM  method  performed  better  than  the  CD  method  especially 
when  the  percent  missing  data  was  large  (Little,  1979) . 

2.  Based  on  bias  of  the  regression  coefficients,   (^-jS) , 
the  CD  method  performed  just  as  well  as  the  MR  and  RG 
methods  when  the  average  correlation  of  the  variables  was 
low  (R=0.2).  When  the  average  correlation  was  high  (R=0.9), 
the  MR  method  was  superior  to  the  CD  and  RG  methods  (Hill  & 
Ziemer,   1983) . 

3.  The  CD,  EM  and  PP  methods  performed  similarly 
regardless  of  the  correlational  structure  when  the  percent 
missing  data  was  low  (Azen  et  al.,   1989).  When  the  percent 
missing  data  was  high,  the  EM  method  performed  better  than 
the  rest. 

The  results  of  the  present  study  have  shown  that 

1.  When  the  number  of  independent  variables  was  low, 
all  the  estimation  methods  performed  similarly  regardless  of 
the  percent  missing  data  or  the  average  correlation.  When 
the  number  of  independent  variables  was  high,    (a)  all  the 
estimation  methods  performed  similarly  when  the  average 
correlation  was  low,  and  (b)  the  CD,  EM  and  MR  methods 
performed  similarly  and  were  the  best  when  the  average 
correlation  was  high,  or  regardless  of  the  percent  missing 
data. 

2.  The  performance  of  all  the  estimation  methods 
declined  as  the  percent  missing  data  increased  from  5%  to 
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15%  regardless  of  (a)  the  number  of  independent  variables, 
with  the  trend  being  nonsignificant  for  the  EM  and  MR 
methods,  and  for  the  PP  method  when  p=6,  or  (b)  the  average 
correlation,  with  the  trend  being  significant  only  for  the 
CD  method,  and  for  the  RG  method  when  the  average 
correlation  was  high. 

3 .  The  performance  of  all  the  estimation  methods 
declined  as  the  average  correlation  increased  from  0.2  to 
0.7  (a)  regardless  of  the  nximber  of  independent  variables, 
however  when  the  number  of  independent  variables  was  low, 
this  trend  was  not  significantly  for  the  PP  and  RG  methods, 
and  (b)   for  all  types  of  missing  data,  though  not 
significantly  except  for  the  RG  method  and  for  the  PP  method 
when  data  were  missing  at  random. 

Thus,  the  present  study  (a)  verifies  the  superiority  of 
the  EM  method,    (b)   shows  that  among  the  ad  hoc  methods,  the 
CD  and  MR  methods  are  as  good  as  the  EM  method  in  most 
situations,  and  (c)   shows  that  the  estimation  is  poorer  when 
data  are  censored  than  when  data  are  missing  at  random. 

Based  on  the  Estimation  of  the  Conditional  Variance 
Very  few  studies  have  used  this  criterion  for  assessing 
the  performance  of  the  estimation  methods.  Beale  and  Little 
(1975)   found  that  the  EM  and  RG  method  consistently  gave 
better  estimate  of  the  residual  than  the  CD  method,   and  that 
the  EM  method  improved  on  the  RG  method. 
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The  results  of  the  present  study  have  shown  that 

1.  The  EM,  MR  and  PP  methods  performed  similarly  and 
were  the  best  when  data  were  normally  distributed.  When  data 
were  lognormally  distributed,  the  EM  and  MR  outperformed  the 
rest.  The  impact  of  skewness  on  the  estimation  methods  was 
negligible  except  for  the  PP  method  whose  performance 
declined  with  increase  in  skewness. 

2.  When  the  percent  missing  data  was  low,  all  the 
estimation  methods  performed  similarly.  When  the  percent 
missing  data  was  high,  the  EM  and  MR  methods  were  superior 
to  the  rest.  However,  the  performance  of  all  the  methods 
declined  when  the  percent  missing  data  increased  from  5%  to 
15%. 

3 .  The  estimation  improved  with  increase  in  sample 
size,  particularly  when  the  percent  missing  data  was  high. 

4.  All  the  estimation  methods  performed  similarly 
regardless  of  the  level  combination  of  p  and  R  except  when 
p=6  and  R=0.7  the  PP  and  RG  methods  were  inferior  to  the 
rest. 

Based  on  the  Estimation  of  the  V-C  matrix 

  1 

This  criterion  was  used  in  the  early  work  of  Timm 
(1970),   and  Gleason  and  Staelin  (1975).  From  Timm  (1970), 
for  the  estimation  of  the  V-C  matrices,  the  results  favored 
the  RG  method  over  the  CD  and  MR  method  when  data  were 
randomly  missing.  No  clear  pattern  by  the  estimation  methods 
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was  noted  as  the  intercorrelation,  percent  missing  data, 
number  of  variables,  or  sample  size  varied. 

From  Gleason  and  Staelin  (1975) ,  for  the  estimation  of 
the  correlation  matrices,  the  results  showed  that 

1.  When  the  average  correlation  was  low  (R<0.2)  the  CD 
method  performed  better  than  the  RG,  MR  and  PP  methods. 

2.  When  R>0.2  the  RG  method  performed  better  than  the 

rest. 

The  results  of  the  present  study  have  shown  that: 

1.  The  EM  method  was  the  best  regardless  of  the 
presence  of  the  other  factors. 

2 .  The  estimation  was  better  when  data  were  missing  at 
random  than  when  data  were  censored  with  right-censored  data 
being  the  poorest. 

3.  The  estimation  declined  as  the  conditional  variance, 
percent  missing  data,  or  average  correlation  increased. 

4.  The  estimation  improved  when  the  number  of 
independent  variables  increased  only  when  the  average 
correlation  was  low. 

The  results  of  the  present  study  seem  to  point  out  one 
other  important  feature  of  the  criteria,  that  the  criterion 
based  on  the  estimation  of  the  regression  coefficients  did 
not  discriminate  the  estimation  methods  as  much  as  the  other 
criteria  did  (Table  27) .  Probably  this  was  due  to  that  for 
the  BSE  criterion  the  estimation  methods  were  interpreted 
from  the  three-way  interactions  only. 
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Suggestions  to  Future  Research 
Based  on  the  Q,  BSE,  or  RE  criteria,  estimation 
accuracy  declined  substantially  when  data  were  censored;  the 
CD,  MR  and  PP  methods  were  most  affected.  However,  when  data 
were  missing  at  random,  the  CD,  MR  and  PP  methods  were  close 
contenders  to  the  EM  method.  This  suggests  that  future 
research  should  investigate  the  degree  of  censoring  the 
estimation  methods  will  tolerate  and  also  investigate  the 
impact  of  other  conditions  in  which  data  are  not  missing  at 
random . 

Second,  the  results  have  shown  that  the  effect  of 
skewness  is  minimal  except  when  estimation  of  missing  values 
is  the  criterion,  while  the  effect  of  R  or  PM  is  very 
strong.  Thus,  it  seems  that  further  research  on  SK  may  not 
be  as  critical  as  studying  a  wider  variety  of  levels  of  R  or 
PM.  In  this  study  the  choice  of  the  structure  of  the 
variables  correlation  matrix  was  arbitrary  as  long  as  R  was 
either  0.2  or  0.7.  Further  research  is  needed  to  investigate 
the  impact  of  varying  structures  of  the  correlation  matrix. 
Though  situations  with  R>0.7  may  be  unrealistic  due  to 
multicollinearity,   it  is  important  to  investigate  the  level 
at  which  multicollinearity  becomes  a  limiting  factor  in  the 
ability  of  the  estimation  methods. 

Third,  sizes  N=100  and  300  were  included  in  the  study. 
The  effect  of  the  sample  size  on  the  estimation  methods  was 
not  strong.   It  may  be  that  for  p=3  and  6,   and  PM=5  and  15, 


126 

N=100  was  sufficiently  high.  Further  research  might  include 
smaller  N  values  for  these  values  of  p  and  PM  and  larger  N 
values  for  larger  values  of  p  and  PM. 

The  superiority  of  the  EM  method  over  all  the  indirect 
methods,  suggests  that  future  research  should  focus  on 
estimation  methods  that  utilize  the  iterative  procedures. 
The  advantages  of  estimating  missing  values  through 
iteration  have  been  demonstrated  by  the  performance  of  the 
EM  method.  Other  possible  iterative  type  of  procedures 
include  (a)  Dear's  principal  component  method,    (b)  the 
general  iterative  principal  component  method,  and  (c)  the 
singular  value  decomposition  method. 


APPENDIX 

Table  28  shows  abbreviations  used.  Table  29  presents 
the  Tukey  calculated  critical  values  at  a=0.05.  Table  30 
presents  the  D,  RE,  BSE,  and  Q  criteria  scores. 
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Table  28 
Abbreviations 


ADDreviauion 

CLib 

f-n        1   n  ^1         1  A  S  ^            ^          1  S  V*  A  C* 

CXdSSlCdX    XSaSX.  SquaJTeS 

r  UK 

^  1  "v*  f  4"  ^  /~\"y*/^  /-V  -y-     >"            Q  eel 

illX so  Xiiy    d  U    X  diiuuiu 

i.'L/iiiu X c L.C X y    d  L>  Xdiiuwiii 

MT. 

Xllu  A.  XlUUILl      X  Xrk.C  X  Xil 

MR 

MZOR 

modified  zero— order  regression 

PC 

^/X  Xill^XL^ClX      IwV^lllk/ IC^l  I  CO 

pp 

^  wxiijk/  X  c  cc           X  ^      V  ax  X  ci     x  c    Vk/i  i  x  y 

RG 

recfre<^<=5i  on  tpdI  acptnpnt 

1  o udx  ox  6o uxiLid ueci  m^dii  oqudx  e  componen us 

TSR 

L-W^     OL>d<^C  XcUXCOOXUll 

V-C 

variance-covariance 

ZOR 

zero-order  regression 

LS 

least  squares 

PML 

pairwise  maximum  likelihood 

MPQ 

mean  pivots  of  the  regression  parameters 

MSPQ 

mean  sum  of  squares  of  the  pivots 
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Table  29 

Tukev  HSD  Critical  Values  at  a=0.05 


Effect  Q  D  RE  BSE 


SK,  R, 

E 

,   N,  p 

0. 

761 

0. 

022 

0. 

020 

0. 

039 

,Nxp 

1. 

076 

0. 

030 

0. 

028 

0. 

055 

TM 

0. 

394 

0. 

009 

0. 

013 

0. 

033 

PM 

0. 

269 

0. 

006 

0. 

009 

0. 

022 

M 

0. 

394 

0. 

013 

0. 

020 

0. 

049 

TMxM 

- 

simple 

effect 

of 

M 

0. 

683 

0. 

023 

0. 

035 

0. 

085 

- 

simple 

effect 

of 

TM 

0. 

683 

0. 

019 

0. 

030 

0. 

073 

TMxPM 

- 

simple 

effect 

of 

PM 

0. 

467 

0. 

010 

0. 

016 

0. 

039 

- 

simple 

effect 

of 

TM 

0. 

558 

0. 

012 

0. 

019 

0. 

046 

PMxM 

- 

simple 

effect 

of 

PM 

0. 

467 

0. 

013 

0. 

020 

0. 

050 

simple 

effect 

of 

M 

0. 

558 

0. 

019 

0. 

028 

0. 

070 

AxTM 

simple 

effect 

of 

TM 

0. 

558 

0. 

012 

0. 

019 

0. 

046 

simple 

effect 

of 

A 

0. 

467 

0. 

010 

0. 

016 

0. 

039 

AxPM 

simple 

effect 

of 

A 

0. 

381 

0. 

008 

0. 

013 

0. 

032 

simple 

effect 

of 

PM 

0. 

381 

0. 

008 

0. 

013 

0. 

032 

AxM 

simple 

effect 

of 

A 

0. 

467 

0. 

013 

0. 

020 

0. 

050 

simple 

effect 

of 

M 

0. 

558 

0. 

019 

0. 

028 

0. 

070 

AxBxTM 

simple 

effect 

of 

A  or  B 

0. 

660 

0. 

015 

0. 

022 

0. 

055 

simple 

effect 

of 

TM 

0. 

789 

0. 

017 

0. 

027 

0. 

065 
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Table  29 — continued. 


Effect  Q  D  RE  BSE 


AxBxM 

simple 

effect 

of 

A  or 

B 

0. 

660 

0. 

019 

0. 

029 

0. 

071 

simple 

effect 

of 

M 

0. 

920 

0. 

026 

0. 

040 

0. 

098 

AxTMxM  - 

simple 

effect 

of 

A 

0. 

808 

0. 

023 

0. 

035 

0. 

086 

simple 

effect 

of 

TM 

0. 

966 

0. 

028 

0. 

042 

0. 

103 

simple 

effect 

of 

M 

1. 

126 

0. 

032 

0. 

049 

0. 

120 

AxPMxM  - 

simple 

effect 

of 

A 

0. 

660 

0. 

019 

0. 

029 

0. 

071 

simple 

effect 

of 

PM 

0. 

660 

0. 

019 

0. 

029 

0. 

071 

simple 

effect 

of 

M 

0. 

920 

0. 

026 

0. 

040 

0. 

098 

AxBxTMxM- 

simple 

effect 

of 

A  or 

B 

1. 

143 

0. 

033 

0. 

050 

0. 

122 

simple 

effect 

of 

TM 

1. 

366 

0. 

039 

0. 

060 

0. 

146 

simple 

effect 

of 

M 

1. 

592 

0. 

045 

0. 

069 

0. 

170 

Note.  A,  B  =  SK  or  R  or  E  or  N  or  p 
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Table  30 

The  BSE.  D.  0.   and  RE  Criteria  Scores 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 


0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 


0 

0 


LC 

LC 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 


6 
6 
6 
6 


12 
12 
12 
12 


6 
6 
6 
6 


5 
5 
5 
5 


6  15 

6  15 

6  15 

6  15 


5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 


100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 


0.025 
0.005 
0.036 
0.014 
0.171 
0.012 
0.138 
0.  041 
0.042 
0.018 
0.061 
0.020 
0.339 
0.083 
0.294 
0.  094 
0.  024 
0.  007 
0.  024 
0.  083 


0.501 
0.441 
0.  350 
0.302 
0.922 
0.655 
0.599 
0.474 
0.  678 
0.562 
0.661 
0.466 
1.469 
0.893 
1.173 
0.753 
0.  648 
0.909 
0.444 
0.  569 


0.  000 
0.000 
0.000 
0.  000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.  000 
0.  000 
0.000 
0.  000 
0.  000 
0.  000 
0.  000 
0.  000 


1.016 
0.976 
0.996 
1.030 
1.163 
1.103 
1.064 
1.102 
1.027 
1.002 
1.016 
1.016 
1. 192 
1. 182 
1.094 
1.097 
0.973 
0.968 
1.010 
0.990 
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Table  3  0 — continued. 


M 

SK 

TM 

R 

E 

PM 

N 

P 

BSE 

D 

Q 

RE 

CD 

0 

LC 

0 . 

7 

6 

15 

100 

3 

0 . 182 

1 . 

173 

0 . 

000 

1.123 

CD 

0 

LC 

0 . 

7 

6 

15 

100 

6 

0 .  055 

1 . 

377 

0 . 

000 

1 .  032 

CD 

0 

LC 

0. 

7 

6 

15 

300 

3 

0 . 139 

0 . 

803 

0 . 

000 

1 .  087 

CD 

0 

LC 

0 . 

7 

6 

15 

300 

6 

0 .  086 

0 . 

912 

0 . 

000 

1.032 

CD 

0 

LC 

0 . 

7 

12 

5 

100 

3 

0 .  029 

0 . 

845 

0 . 

000 

1 .  007 

CD 

0 

LC 

0. 

7 

12 

5 

100 

6 

0.  028 

0. 

918 

0 . 

000 

0.  953 

CD 

0 

LC 

0 . 

7 

12 

5 

300 

3 

0 . 036 

0 . 

731 

0. 

000 

1 .  006 

CD 

0 

LC 

0 . 

7 

12 

5 

300 

6 

0 .  048 

0 . 

674 

0 . 

000 

0.  986 

CD 

0 

LC 

0 . 

7 

12 

15 

100 

3 

0.366 

1 . 

777 

0 . 

000 

1 . 166 

CD 

0 

LC 

0  . 

7 

12 

15 

100 

6 

0.290 

1 . 

544 

0 . 

000 

1 .  047 

r\ 
U 

L»C 

0  • 

7 

12 

15 

1 

300 

3 

0.251 

1 . 

288 

0 . 

000 

1 .  085 

or* 
CU 

0 

LC 

0 . 

7 

12 

15 

O 

300 

6 

0 . 102 

1 . 

085 

0 . 

000 

1.035 

CD 

0 

RC 

0 . 

2 

6 

5 

100 

3 

0 .  023 

0 . 

435 

0 . 

000 

1 .  017 

CD 

0 

RC 

0  . 

2 

6 

5 

100 

6 

0.009 

0 . 

468 

0 . 

000 

0 .  984 

CD 

0 

RC 

0 . 

2 

6 

5 

300 

3 

0 .  028 

0 . 

358 

0 . 

000 

0 . 995 

CD 

0 

RC 

0. 

2 

6 

5 

300 

6 

0.019 

0. 

321 

0. 

000 

1.  034 

CD 

0 

RC 

0. 

2 

6 

15 

100 

3 

0.204 

0. 

818 

0. 

000 

1.200 

CD 

0 

RC 

0. 

2 

6 

15 

100 

6 

0.  039 

0. 

711 

0. 

000 

1. 179 

CD 

0 

RC 

0. 

2 

6 

15 

300 

3 

0.134 

0. 

618 

0. 

000 

1.  066 

CD 

0 

RC 

0. 

2 

6 

15 

300 

6 

0.067 

0. 

519 

0. 

000 

1. 118 

■  I 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 


0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


12 
12 
12 
12 
12 
12 
12 
12 
6 
6 
6 
6 
6 
6 
6 
6 
12 
12 
12 
12 


5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 


3 
6 
3 


3  0.034 
6  0.027 
0.041 
0.036 
0.412 
6  0.141 
3  0.236 
6  0.161 
3  0.008 
6  0.004 
3  0.023 
6  0.188 
0.136 
0.017 
0. 132 
6  0.266 
3  0.041 
6  0.019 
3  0.076 


3 
6 
3 


5     300     6  0.085 


0.627 
0.586 
0.640 
0.478 
1.308 
0.937 
1.116 
0.803 
0.548 
0.859 
0.469 
0.566 
1.085 
1.331 
0.819 
0.884 
0.886 
0.  947 
0.746 
0.703 


0.000 
0.000 
0.000 
0.  000 
0.000 
0.  000 
0.000 
0.000 
0.000 
0.  000 
0.  000 
0.000 
0.000 
0.  000 
0.  000 
0.  000 
0.  000 
0.000 
0.  000 
0.  000 


1.021 
0.992 
1.015 
1.016 
1.198 
1.195 
1.086 
1.121 
0.964 

0.  961 

1.  014 

0.  990 
1.085 

1.  031 
1.093 
1.025 
1.  008 

0.  956 

1.  006 
0.986 
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Table  3  0 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

CD  0  RC  0.7  12  15  100  3  0.336  1.736  0.000  1.162 

CD  0  RC  0.7  12  15  100  6  0.095  1.579  0.000  1.046 

CD  0  RC  0.7  12  15  300  3  0.325  1.318  0.000  1.079 

CD  0  RC  0.7  12  15  300  6  0.219  1.097  0.000  1.038 

CD  0  RM  0.2  6  5  100  3  0.001  0.156  0.000  0.996 

CD  0  RM  0.2  6  5  100  6  0.001  0.160  0.000  0.965 

CD  0  RM  0.2  6  5  300  3  0.002  0.060  0.000  0.979 

CD  0  RM  0.2  6  5  300  6  0.002  0.069  0.000  1.019 

CD  0  RM  0.2  6  15  100  3  0.005  0.393  0.000  1.054 

CD  0  RM  0.2  6  15  100  6  0.005  0.374  0.000  1.140 

CD  0  RM  0.2  6  15  300  3  0.003  0.113  0.000  0.989 

CD  0  RM  0.2  6  15  300  6  0.009  0.171  0.000  1.067 

CD  0  RM  0.2  12  5  100  3  0.000  0.207  0.000  1.002 

CD  0  RM  0.2  12  5  100  6  0.002  0.243  0.000  0.974 

CD  0  RM  0.2  12  5  300  3  0.001  0.119  0.000  0.999 

CD  0  RM  0.2  12  5  300  6  0.006  0.119  0.000  0.999 

CD  0  RM  0.2  12  15  100  3  0.010  0.671  0.000  1.079 

CD  0  RM  0.2  12  15  100  6  0.007  0.608  0.000  1.138 

CD  0  RM  0.2  12  15  300  3  0.001  0.262  0.000  1.008 

CD  0  RM  0.2  12  15  300  6  0.007  0.251  0.000  1.046 
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Table  30 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


RE 


CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 


0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 


2 
2 
2 
2 


LC 
LC 
LC 
LC 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 


6 
6 
12 
12 
12 
12 


6 
6 
6 
6 


5 
5 
5 
5 


6  15 
6  15 


15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.002 

6  0.012 

3  0.002 

6  0.119 

3  0.020 

6  0.584 

3  0.000 

6  0.057 

3  0.001 

6  0.005 

3  0.028 

6  0.188 

3  0.009 
0.046 
0.  014 

6  0.308 

3  0.028 

6  0.008 

3  0.027 

6  0.017 


6 
3 


0. 155 
0.212 
0.081 
0.092 
0.376 
0.546 
0.144 
0.232 
0.204 
0.264 
0. 121 
0. 142 
0.642 
0.  687 
0.253 
0.320 
0.328 
0.  206 
0.320 
0.200 


0.000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.  000 
0.000 
0.000 
0.  000 
0.  000 
0.  000 
0.  000 
0.  000 


0.954 
0.965 
0.999 
0.988 
1.006 
1. 136 
1.010 
1.037 
0.992 
0.960 
0.990 

0.  982 

1.  056 
1. 145 
1.001 
1.027 
0.982 
0.  953 
0.995 
0.994 
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Table  30 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

CD  2  LC  0.2  6  15  100  3  0.144  0.571  0.000  1.067 

CD  2  LC  0.2  6  15  100  6  0.034  0.344  0.000  1.052 

CD  2  LC  0.2  6  15  300  3  0.135  0.548  0.000  1.064 

CD  2  LC  0.2  6  15  300  6  0.070  0.305  0.000  1.042 

CD  2  LC  0.2  12  5  100  3  0.052  0.618  0.000  0.962 

CD  2  LC  0.2  12  5  100  6  0.046  0.369  0.000  0.971 

CD  2  LC  0.2  12  5  300  3  0.051  0.584  0.000  1.025 

CD  2  LC  0.2  12  5  300  6  0.021  0.368  0.000  1.027 

CD  2  LC  0.2  12  15  100  3  0.267  1.096  0.000  1.045 

CD  2  LC  0.2  12  15  100  6  0.171  0.600  0.000  1.095 

CD  2  LC  0.2  12  15  300  3  0.275  1.090  0.000  1.099 

CD  2  LC  0.2  12  15  300  6  0.126  0.606  0.000  1.103 

CD  2  LC  0.7  6  5  100  3  0.020  0.301  0.000  0.973 

CD  2  LC  0.7  6  5  100  6  0.026  0.289  0.000  0.977 

CD  2  LC  0.7  6  5  300  3  0.034  0.388  0.000  1.025 

CD  2  LC  0.7  6  5  300  6  0.019  0.248  0.000  0.976 

CD  2  LC  0.7  6  15  100  3  0.093  0.553  0.000  1.036 

CD  2  LC  0.7  6  15  100  6  0.041  0,453  0.000  1.036 

CD  2  LC  0.7  6  15  300  3  0.129  0.615  0.000  1.092 

CD  2  LC  0.7  6  15  300  6  0.044  0.379  0.000  1.004 
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Table  3  0 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

CD  2  LC  0.7  12  5  100  3  0.014  0.562  0.000  0.932 

CD  2  LC  0.7  12  5  100  6  0.025  0.427  0.000  0.972 

CD  2  LC  0.7  12  5  300  3  0.047  0.678  0.000  1.017 

CD  2  LC  0.7  12  5  300  6  0.028  0.435  0.000  0.977 

CD  2  LC  0.7  12  15  100  3  0.229  1.090  0.000  1.009 

CD  2  LC  0.7  12  15  100  6  0.280  0.692  0.000  1.052 

CD  2  LC  0.7  12  15  300  3  0.247  1.175  0.000  1.085 

CD  2  LC  0.7  12  15  300  6  0.141  0.682  0.000  1.019 

CD  2  RC  0.2  6  5  100  3  0.041  0.464  0.000  1.025 

CD  2  RC  0.2  6  5  100  6  0.018  0.440  0.000  1.034 

CD  2  RC  0.2  6  5  300  3  0.029  0.453  0.000  1.007 

CD  2  RC  0.2  6  5  300  6  0.008  0.467  0.000  1.020 

CD  2  RC  0.2  6  15  100  3  0.103  0.706  0.000  1.195 

CD  2  RC  0.2  6  15  100  6  0.095  0.620  0.000  1.455 

CD  2  RC  0.2  6  15  300  3  0.149  0.714  0.000  1.092 

CD  2  RC  0.2  6  15  300  6  0.055  0.667  0.000  1.156 

CD  2  RC  0.2  12  5  100  3  0.060  0.653  0,000  0.990 

CD  2  RC  0.2  12  5  100  6  0.019  0.578  0.000  1.042 

CD  2  RC  0.2  12  5  300  3  0.035  0.701  0.000  1.033 

CD  2  RC  0.2  12  5  300  6  0.024  0.579  0.000  1.048 


138 


Table  30 — continued. 


M 

SK 

TM 

R 

E  • 

PM 

N 

P 

BSE 

D 

Q 

RE 

CD 

2 

RC 

0 . 

2 

12 

15 

100 

3 

0.373 

1 . 

136 

0 . 

000 

1 . 157 

CD 

2 

RC 

0 . 

2 

12 

15 

100 

6 

0 . 154 

0 . 

857 

0 . 

000 

1.423 

CD 

2 

RC 

0 . 

2 

12 

15 

300 

3 

0.234 

1 . 

206 

0 . 

f\  f\  f\ 

000 

1 . 131 

CD 

2 

RC 

0 . 

2 

12 

15 

300 

6 

0.133 

0 . 

898 

0 . 

000 

1.191 

CD 

2 

RC 

0 . 

7 

6 

5 

100 

3 

0 . 025 

0 . 

637 

0 . 

000 

1 . 006 

CD 

2 

RC 

0. 

7 

6 

5 

100 

6 

0. 153 

0. 

917 

0 . 

000 

1 . 175 

CD 

2 

RC 

0. 

7 

6 

5 

300 

3 

0 . 028 

0 . 

635 

0 . 

000 

1 .  Oil 

CD 

2 

RC 

0 . 

7 

6 

5 

300 

6 

0 . 004 

0 . 

921 

0 . 

000 

0 . 996 

CD 

2 

RC 

0 . 

7 

6 

15 

100 

3 

0.128 

0 . 

r\  o 

992 

0  • 

000 

1 .  160 

CD 

2 

RC 

0 . 

7 

6 

15 

100 

6 

0 .  050 

1  • 

244 

r\ 

0  • 

r\  r\  r\ 

000 

1 .  4  U4 

CD 

2 

RC 

0  . 

7 

6 

15 

o 

300 

3 

0  •  13  7 

r\ 

0  • 

y  bo 

000 

1  .  lUU 

CD 

2 

RC 

0 . 

7 

6 

15 

3  00 

6 

0 .  150 

1 

1 . 

243 

0 . 

r\  r\  r\ 

000 

1.0// 

CD 

2 

RC 

0 . 

7 

12 

5 

100 

3 

0 .  049 

0 . 

820 

0 . 

000 

0.964 

CD 

2 

RC 

0 . 

7 

12 

5 

100 

6 

1.535 

1 . 

120 

0 . 

000 

1.061 

CD 

2 

RC 

0 . 

7 

12 

5 

300 

3 

0.056 

0 . 

895 

0 . 

000 

1.029 

CD 

2 

RC 

0. 

7 

12 

5 

300 

6 

0.040 

1. 

067 

0. 

000 

0.998 

CD 

2 

RC 

0. 

7 

12 

15 

100 

3 

0.272 

1. 

401 

0. 

000 

1. 124 

CD 

2 

RC 

0. 

7 

12 

15 

100 

6 

1.012 

1. 

544 

0. 

000 

1.298 

CD 

2 

RC 

0. 

7 

12 

15 

300 

3 

0.291 

1. 

442 

0. 

000 

1. 125 

CD 

2 

RC 

0. 

7 

12 

15 

300 

6 

0. 116 

1. 

478 

0. 

000 

1.  096 
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Table  3  0 — continued. 


M 

SK 

TM 

R 

E 

PM 

N 

P 

BSE 

D 

Q 

RE 

CD 

2 

RM 

0. 

2 

6 

5 

100 

3 

0.  001 

0. 

127 

0. 

000 

0.971 

CD 

2 

RM 

0. 

2 

6 

5 

100 

6 

0.007 

0. 

143 

0. 

000 

0.950 

CD 

2 

RM 

0. 

2 

6 

5 

300 

3 

0.001 

0. 

074 

0. 

000 

0.981 

CD 

2 

RM 

0. 

2 

6 

5 

300 

6 

0.011 

0. 

098 

0. 

000 

0.989 

CD 

2 

RM 

0. 

2 

6 

15 

100 

3 

0.004 

0. 

245 

0. 

000 

1.004 

CD 

2 

RM 

0. 

2 

6 

15 

100 

6 

0.005 

0. 

307 

0. 

000 

1.154 

CD 

2 

RM 

0. 

2 

6 

15 

300 

3 

0.001 

0. 

148 

0. 

000 

0.991 

CD 

2 

RM 

0. 

2 

6 

15 

300 

6 

0.018 

0. 

204 

0. 

000 

1.036 

CD 

2 

RM 

0. 

2 

12 

5 

100 

3 

0.002 

0. 

212 

0. 

000 

0.947 

CD 

2 

RM 

0. 

2 

12 

5 

100 

6 

0.005 

0. 

218 

0. 

000 

0.970 

CD 

2 

RM 

0. 

2 

12 

5 

300 

3 

0.001 

0. 

121 

0. 

000 

1.012 

CD 

2 

RM 

0. 

2 

12 

5 

300 

6 

0.001 

0. 

117 

0. 

000 

1.016 

CD 

2 

RM 

0. 

2 

12 

15 

100 

3 

0.018 

0. 

418 

0. 

000 

0.993 

CD 

2 

RM 

0. 

2 

12 

15 

100 

6 

0.020 

0. 

479 

0. 

000 

1.179 

CD 

2 

RM 

0. 

2 

12 

15 

300 

3 

0.005 

0. 

227 

0. 

000 

1.021 

CD 

2 

RM 

0. 

2 

12 

15 

300 

6 

0.007 

0. 

308 

0. 

000 

1.071 

CD 

2 

RM 

0. 

7 

6 

5 

100 

3 

0.001 

0. 

135 

0. 

000 

0.966 

CD 

2 

RM 

0. 

7 

6 

5 

100 

6 

0.255 

0. 

258 

0. 

000 

1.003 

CD 

2 

RM 

0. 

7 

6 

5 

300 

3 

0.  015 

0. 

084 

0. 

000 

1.013 

CD 

2 

RM 

0. 

7 

6 

5 

300 

6 

0.005 

0. 

124 

0. 

000 

0.975 
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Table  3  0 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
CD 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 


2  RM 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


RM 
RM 
RM 
RM 
RM 
RM 
RM 
RM 
RM 
RM 
RM 


0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
12 
12 
12 
12 


6 
6 
6 
6 


15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 


6  15 
6  15 


6 
6 


15 
15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.004 

6  0.363 

3  0.009 

6  0.095 

3  0.005 

6  0.012 

3  0.002 

6  0.003 

3  0.077 

6  0.229 

3  0.005 

6  0.099 

3  0.009 

6  0.004 

3  0.021 

6  0.008 

3  0.032 

6  0.019 

3  0.068 

6  0.017 


0.268 
0.454 
0.187 
0.297 
0.176 
0.275 
0.139 
0.180 
0.456 
0.745 
0.252 
0.408 
0.452 
0.331 
0.339 
0.261 
0.807 
0.  646 
0.  611 
0.512 


0.  000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.000 
2.323 
0.699 
2  . 199 
0.741 
2.486 
0.827 
1.966 
0.  850 


1.001 
1.298 
1.021 
1.027 
0.926 
0.991 
1.003 
0.974 
0.965 
1.219 
1.012 
1.032 
1.009 
0.967 

0.  985 
1.018 
1.073 

1.  030 
1.  015 
1.  044 
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Table  3  0 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 


0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


12 
12 
12 
12 


6 
6 
6 
6 
6 
6 


12 
12 
12 
12 


5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 
15 
15 


6  15 
6  15 


5 
5 
5 
5 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3 
6 
3 
6 
3 


0.017 
0.004 
0.  034 
0.  002 
0. 116 
6  0.009 
3  0.137 
6  0.021 
3  0.008 
6  0.010 
3  0.012 
6  0.032 
0.007 
0.042 
3  0.031 
6  0.072 
3  0.013 
6  0.028 
3  0.016 
6  0.019 


3 
6 


0.  666 
0.502 
0.643 

0.  441 

1.  378 
0.932 
1.142 
0.828 
0.469 
0.434 
0.355 
0.341 
0.  650 
0.767 
0.573 
0.704 
0.727 
0.538 
0.664 
0.501 


2.349 
0.756 
2.330 
0.808 
2.767 
0.878 
1.956 
0.874 
1.744 
0.218 
1.820 
0.233 
1.497 
0.292 
1.256 
0.317 
1.780 
0.244 
2.016 
0.261 


1.017 
0.970 
1.004 
0.999 
1.084 
1.028 
1.039 
1.023 
0.965 
0.954 
1.003 
0.986 
0.995 

0.  984 
1.018 
1.017 

1.  003 
0.944 
1.003 
0.983 
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Table  30 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

EM  0  LC  0.7  12  15  100  3  0.065  1.241  1.848  1.055 

EM  0  LC  0.7  12  15  100  6  0.169  1.052  0.356  0.975 

EM  0  LC  0.7  12  15  300  3  0.070  1.035  1.479  1.027 

EM  0  LC  0.7  12  15  300  6  0.082  1.014  0.334  1.027 

EM  0  RC  0.2  6  5  100  3  0.009  0.421  2.349  1.010 

EM  0  RC  0.2  6  5  100  6  0.003  0.352  0.716  0.965 

EM  0  RC  0.2  6  5  300  3  0.015  0.342  2.313  0.985 

EM  0  RC  0.2  6  5  300  6  0.007  0.272  0.768  1.021 

EM  0  RC  0.2  6  15  100  3  0.059  0.826  2.585  1.081 

EM  0  RC  0.2  6  15  100  6  0.015  0.694  0.828  1.041 

EM  0  RC  0.2  6  15  300  3  0.058  0.613  1.942  1.011 

EM  0  RC  0.2  6  15  300  6  0.018  0.535  0.910  1.045 

EM  0  RC  0.2  12  5  100  3  0.011  0.630  2.402  1.015 

EM  0  RC  0.2  12  5  100  6  0.006  0.490  0.760  0.968 

EM  0  RC  0.2  12  5  300  3  0.019  0.626  2.253  1.004 

EM  0  RC  0.2  12  5  300  6  0.011  0.457  0.787  1.000 

EM  0  RC  0.2  12  15  100  3  0.133  1.353  2.723  1.088 

EM  0  RC  0.2  12  15  100  6  0.024  0.943  0.887  1.017 

EM  0  RC  0.2  12  15  300  3  0.112  1.143  1.923  1.039 

EM  0  RC  0.2  12  15  300  6  0.050  0.859  0.894  1.032 


143 


Table  30 — continued. 


M  SK  TM  R  E  PM  N    .  p  BSE  D  Q  RE 

EM  0  RC  0.7  6  5  100  3  0.004  0.409  1.608  0.963 

EM  0  RC  0.7  6  5  100  6  0.009  0.381  0.204  0.951 

EM  0  RC  0.7  6  5  300  3  0.009  0.362  1.506  1.006 

EM  0  RC  0.7  6  5  300  6  0.169  0.329  0.207  0.986 

EM  0  RC  0.7  6  15  100  3  0.008  0.682  1.652  0.989 

EM  0  RC  0.7  6  15  100  6  0.013  0.750  0.262  0.983 

EM  0  RC  0.7  6  15  300  3  0.021  0.559  1.238  1.019 

EM  0  RC  0.7  6  15  300  6  0.383  0.696  0.314  1.018 

EM  0  RC  0.7  12  5  100  3  0.017  0.727  1.602  1.002 

EM  0  RC  0.7  12  5  100  6  0.093  0.520  0.230  0.943 

EM  0  RC  0.7  12  5  300  3  0.048  0.681  2.018  1.003 

EM  0  RC  0.7  12  5  300  6  0.046  0.507  0.261  0.983 

EM  0  RC  0.7  12  15  100  3  0.104  1.157  1.717  1.048 

EM  0  RC  0.7  12  15  100  6  0.009  0.974  0.320  0.978 

EM  0  RC  0.7  12  15  300  3  0.137  1.077  1.436  1.026 

EM  0  RC  0.7  12  15  300  6  0.087  1.006  0.372  1.024 

EM  0  RM  0.2  6  5  100  3  0.000  0.087  0.874  0.995 

EM  0  RM  0.2  6  5  100  6  0.000  0.056  0.129  0.942 

EM  0  RM  0.2  6  5  300  3  0.003  0.040  0.926  0.977 

EM  0  RM  0.2  6  5  300  6  0.003  0.028  0.130  1.012 
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Table  3  0 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


D 


Q 


RE 


EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 


0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


6 
6 
6 
6 
12 
12 
12 
12 


6 
6 
6 
6 
6 
6 


15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 
15 
15 


100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 


100 

300 


6 

3 


6  15 
6  15 


300  6 

100  3 

100  6 

300  3 

300  6 


0.000 
0.001 
0.005 
0.005 
0.000 
0.001 
0.001 
0.003 
0.000 
0.002 
0.000 
0.003 
0.004 
0.004 
0.002 
0.  032 
0.031 
0.  388 
0.002 
0.280 


0.149 
0.103 
0.069 
0.048 
0.104 
0.079 
0.073 
0.045 
0.276 
0.154 
0. 118 
0.080 
0.079 
0.  059 
0.049 
0.024 
0. 140 
0. 126 
0.082 
0.057 


0.614 
0.149 
0.450 
0. 144 
0.830 
0. 144 
0.797 
0. 136 
0.592 
0. 161 
0.478 
0. 153 
0.716 
0.022 
0.698 
0.  023 
0.413 
0.  035 
0.350 
0.  034 


1.012 
0.967 
0.980 
1.019 
1.003 
0.947 
0.994 
0.991 
1.022 
0.966 
0.997 
0.997 
0.955 
0.944 
0.999 
0.981 
0.976 

0.  989 

1.  005 
0.994 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p  BSE 


RE 


EM 


0     RM     0.7  12 


EM       0     RM     0.7  12 


EM 


0     RM     0.7  12 


EM  0  RM  0.7  12 

EM  0  RM  0.7  12 

EM  0  RM  0.7  12 

EM  0  RM  0.7  12 

EM  0  RM  0.7  12 

EM  2  LC  0.2  6 

EM  2  LC  0.2  6 

EM  2  LC  0.2  6 

EM  2  LC  0.2  6 

EM  2  LC  0.2  6 

EM  2  LC  0.2  6 

EM  2  LC  0.2  6 

EM  2  LC  0.2  6 

EM  2  LC  0.2  12 

EM  2  LC  0.2  12 

EM  2  LC  0.2  12 

EM  2  LC  0.2  12 


5  100  3  0.000  0.143  0.717  0.989 

5  100  6  0.031  0.081  0.025  0.935 

5  300  3  0.019  0.072  0.703  0.993 

5  300  6  0.104  0.040  0.026  0.974 

15  100  3  0.004  0.298  0.431  1.012 

15  100  6  0.063  0.148  0.039  0.965 

15  300  3  0.016  0.112  0.355  0.997 

15  300  6  0.313  0.081  0.038  0.984 

5  100  3  0.016  0.356  1.287  0.983 

5  100  6  0.004  0.214  0.249  0.933 

5  300  3  0.016  0.331  1.288  0.985 

5  300  6  0.011  0.225  0.229  0.985 

15  100  3  0.089  0.632  1.012  1.031 

15  100  6  0.016  0.443  0.300  0.965 

15  300  3  0.075  0.598  0.942  1.019 

15  300  6  0.037  0.454  0.282  1.005 

5  100  3  0.034  0.607  1.373  0.959 

5  100  6  0.028  0.387  0.259  0.950 

5  300  3  0.033  0.630  1.300  1.024 

5  300  6  0.008  0.414  0.277  1.015 


Table  30 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


D 


RE 


EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


LC 

LC 

LC 

LC 

LC 

LC 

LC 

LC 

LC 

LC 

LC 

LC. 

LC 

LC 

LC 

LC 

LC 

LC 

LC 

LC 


0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 


6 
6 
12 
12 
12 
12 


5 
5 
5 
5 


6  15 
6  15 


15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3 
6 
3 


3  0.184 
6  0.095 
0.172 
0.053 
0.013 
6  0.033 
3  0.027 
6  0.026 
0.051 
0.068 
0.068 
6  0.051 
3  0.031 
0.027 
0.  034 
6  0.023 
3  0.138 
6  0.238 
3  0.141 
6  0.130 


3 
6 
3 


6 
3 


1. 150 
0.771 
1. 182 
0.777 
0.308 
0.282 
0.385 
0.257 
0.544 
0.576 
0.592 
0.529 
0.579 
0.470 
0.  677 
0.472 
0.966 
0.911 
1. 102 
0.903 


1.058 
0.340 
0.971 
0.336 
1.112 
0.096 
17.12 
0.097 
0.724 
0. 124 
5.256 
0.141 
1.135 

0.  133 

1.  156 
0. 125 
0.840 
0. 160 
0.799 
0. 166 


1.012 
0.997 
1.066 
1.048 
0.976 
0.975 
1.024 
0.976 
1.012 
1.033 
1.054 
1.014 
0.935 
0.966 
1.012 
0.977 

0.  980 

1.  031 
1.  049 
1.  029 
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Table  30 — continued. 
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SK 

TM 

R 

E 

PM 
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P 

u 
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EM 
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X  •  U  /  b 

EM 

2 

RC 

0. 

2 

12 

15 

300 

6 

0.  008 

0. 

911 

2 . 982 

1.  065 

EM 

2 

RC 

0. 

7 

6 

5 

100 

3 

0.002 

0. 

408 

3.555 

0.995 

EM 

2 

RC 

0. 

7 

6 

5 

100 

6 

0.079 

0. 

550 

10.06 

1. 147 

EM 

2 

RC 

0. 

7 

6 

5 

300 

3 

0.001 

0. 

407 

3  .976 

1.003 

EM 

2 

RC 

0. 

7 

6 

5 

300 

6 

0.005 

0. 

496 

0.922 

0.986 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 


2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
12 
12 
12 
12 


6 
6 
6 
6 
6 
6 
6 


15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 
15 
15 
15 


6  15 


100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 


0.024 
0.030 
0.015 
0.127 
0.024 
1.198 
0.018 
0.032 
0.040 
0. 118 
0.048 
0.070 
0.000 
0.005 
0.001 
0.004 
0.002 
0.  005 
0.  001 
0.005 


0.611 
0.834 
0.566 
0.874 
0.619 
0.  683 
0.699 
0.705 
0.914 
1.280 
0.978 
1.202 
0.070 
0.  048 
0.042 
0.  028 
0. 127 
0.  097 
0.067 
0.050 


3.874 
4.457 
4.307 
0.913 
1.488 
0.983 
4.515 
0.972 
1.578 
0.997 
4.582 
0.965 
0.844 
0. 155 
0.801 
0. 143 
0.492 
0. 188 
0.465 
0. 151 


1.110 
1.227 
1.036 

1.030 
0.954 
1.028 
1.015 
0.986 
1.061 
1.113 
1.044 
1.034 
0.974 
0.925 
0.976 
0.981 
0.985 
0.959 
0.980 
0.987 
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Table  3  0 — continued. 


M       SK     TM  R 


PM     N         p  BSE 


RE 


EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 
EM 


2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 


0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


6 
6 
6 
6 
6 
6 
6 
6 


0.7  12 

0.7  12 

0.7  12 

0.7  12 


5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 


3  0.001 

6  0.003 

3  0.000 

6  0.000 

3  0.008 

6  0.003 

3  0.001 

6  0.002 

3  0.001 

6  0.200 

3  0.013 

6  0.006 

3  0.009 

6  0.579 

3  0.013 

6  0.090 

3  0.001 

6  0.001 

3  0.002 

6  0.003 


0. 112 
0.069 
0.071 
0.034 
0.211 
0.139 
0.109 
0.077 
0.054 
0.061 
0.040 
0.  026 
0.113 
0.099 
0.060 
0.  046 
0. 102 
0.077 
0.061 
0.040 


0.796 
0.140 
0.976 
0. 146 
0.516 
0.164 
0.521 
0.154 
0.644 
0.031 
0.  668 
0.028 
0.331 
0.039 
0.  332 
0.037 
0.734 
0.034 
0.689 
0.  032 


0.946 
0.942 
1.015 
1.007 
0.961 
0.968 
1.019 
1.015 
0.968 
0.970 
1.011 
0.968 

0.  980 
1.020 

1.  014 
0.  979 
0.925 
0.956 
1.000 
0.965 
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Table  30 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

EM  2  RM  0.7  12  15  100  3  0.031  0.174  0.359  0.935 

EM  2  RM  0.7  12  15  100  6  0.020  0.134  0.050  1.004 

EM  2  RM  0.7  12  15  300  3  0.001  0.098  0.341  1.003 

EM  2  RM  0.7  12  15  300  6  0.057  0.071  0.045  0.977 

MR  0  LC  0.2       6  5  100  3  0.023  0.559  4.761  1.015 

MR  0  LC  0.2       6  5  100  6  0.007  0.454  4.599  0.983 

MR  0  LC  0.2       6  5  300  3  0.030  0.421  4.757  0.990 

MR  0  LC  0.2       6  5  300  6  0.013  0.336  4.772  1.024 

MR  0  LC  0.2       6  15  100  3  0.127  1.077  4.208  1.098 

MR  0  LC  0.2       6  15  100  6  0.023  0.855  3.468  1.045 

MR  0  LC  0.2       6  15  300  3  0.108  0.785  3.469  1.029 

MR  0  LC  0.2       6  15  300  6  0.036  0.658  3.496  1.057 

MR  0  LC  0.2  12  5  100  3  0.030  0.799  4.584  1.019 

MR  0  LC  0.2  12  5  100  6  0.009  0.630  4.566  0.976 

MR  0  LC  0.2  12  5  300  3  0.045  0.798  4.823  1.011 

MR  0  LC  0.2  12  5  300  6  0.008  0.550  4.736  1.003 

MR  0  LC  0.2  12  15  100  3  0.210  1.802  4.211  1.095 

MR  0  LC  0.2  12  15  100  6  0.029  1.189  3.459  1.032 

MR  0  LC  0.2  12  15  300  3  0.194  1.484  3.509  1.053 

MR  0  LC  0.2  12  15  300  6  0.044  1.055  3.481  1.036 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 


0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  RC 

0  RC 

0  RC 

0  RC 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 


12 
12 
12 
12 


5 
5 
5 
5 


6  15 

6  15 

6  15 

6  15 


5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 


5 
5 
5 
5 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.021 

6  0.179 

3  0.013 

6  0.028 

3  0.062 

6  0.228 

3  0.066 

6  0.056 

3  0.019 

6  0.025 
0.027 
0.090 
0.146 

6  0.041 

3  0.122 

6  0.088 

3  0.021 

6  0.009 

3  0.026 

6  0.014 


3 
6 
3 


0.619 
0.719 
0.454 
0.529 
1.010 
1.223 
0.802 
1.040 
0.882 
0.807 
0.803 
0.679 
1.745 
1.514 
1.376 
1.367 
0.520 
0.474 
0.428 
0.347 


4.542 
4.593 
4.637 
4.  662 
3.893 
3.358 
3.323 
3.330 
4.567 
4.473 
4.678 
4.618 
4.053 
3.319 
3.326 

3  .298 
4.579 
4.640 

4  .  756 
4.821 


0.974 
0.988 
1.006 
0.993 
1.034 
1.032 
1.038 
1.027 
1.010 
0.969 
1.000 

0.  987 

1.  075 
1.  003 
1.  037 
1.  020 
1.  013 
0.986 

0.  990 

1.  027 
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Table  3  0 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  ,  Q  RE 

MR  0  RC  0.2       6  15  100  3  0.137  1.091  4.185  1.108 

MR  0  RC  0.2       6  15  100  6  0.029  0.887  3.466  1.063 

MR  0  RC  0.2       6  15  300  3  0.103  0.791  3.456  1.028 

MR  0  RC  0.2       6  15  300  6  0.042  0.677  3.507  1.059 

MR  0  RC  0.2  12  5  100  3  0.019  0.763  4.714  1.013 

MR  0  RC  0.2  12  5  100  6  0.012  0.627  4.610  0.977 

MR  0  RC  0.2  12  5  300  3  0.025  0.776  4.726  1.008 

MR  0  RC  0.2  12  5  300  6  0.019  0.564  4.778  1.005 

MR  0  RC  0.2  12  15  100  3  0.210  1.774  4.254  1.094 

MR  0  RC  0.2  12  15  100  6  0.056  1.201  3.460  1.029 

MR  0  RC  0.2  12  15  300  3  0.155  1.478  3.478  1.050 

MR  0  RC  0.2  12  15  300  6  0.084  1.082  3.511  1.046 

MR  0  RC  0.7       6  5  100  3  0.008  0.564  4.510  0.930 

MR  0  RC  0.7       6  5  100  6  0.196  0.680  4.518  0.989 

MR  0  RC  0.7       6  5  300  3  0.015  0.463  4.638  1.009 

MR  0  RC  0.7       6  5  300  6  0.085  0.518  4.599  0.994 

MR  0  RC  0.7       6  15  100  3  0.031  1.039  3.978  1.010 

MR  0  RC  0.7       6  15  100  6  0.106  1.208  3.335  1.038 

MR  0  RC  0.7       6  15  300  3  0.062  0.787  3.275  1.040 

MR  0  RC  0.7       6  15  300  6  0.086  1.028  3.312  1.026 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 


0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 


0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
6 
6 
6 
6 


0.2  12 

0.2  12 

0.2  12 

0.2  12 


5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 


6 
3 


3  0.019 
6  0.017 
3  0.056 
6  0.045 
3  0.154 
0.037 
0.180 
6  0.081 
3  0.002 
6  0.002 
3  0.004 
6  0.004 
3  0.016 
0.005 
0.  015 
0.  013 
0.  001 
6  0.001 
3  0.003 
6  0.003 


6 
3 
6 
3 


0.881 

0.813 

0.827 

0.  693 

1.683 

1.464 

1.425 

1.369 

0.142 

0. 120 

0.096 

0.  082 

0.472 

0.309 

0.275 

0.236 

0. 160 

0. 149 

0.  182 

0. 127 


4.553 
4  .  688 
4.751 
4.730 
3.956 
3.428 
3.382 
3.354 
0.996 
0.975 
1.004 

0.  991 
1.336 

1.  Oil 
0.989 
0.994 

0.  981 
1.002 

1.  004 
0.988 


1.004 
0.969 
1.003 
0.988 
1.066 
1.010 
1.037 
1.021 
0.992 
0.949 
0.978 
1.013 
1.013 
0.978 

0.  984 

1.  025 
0.999 
0.949 
0.998 
0.991 
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Table  3  0 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

MR  0  RM  0.2  12  15  100  3  0.019  0.773  1.337  1.016 

MR  0  RM  0.2  12  15  100  6  0.008  0.418  1.004  0.972 

MR  0  RM  0.2  12  15  300  3  0.011  0.520  1.006  1.001 

MR  0  RM  0.2  12  15  300  6  0.008  0.356  0.992  1.000 

MR  0  RM  0.7  6  5  100  3  0.001  0.154  0.991  0.952 

MR  0  RM  0.7  6  5  100  6  0.079  0.199  1.017  0.965 

MR  0  RM  0.7  6  5  300  3  0.002  0.115  1.017  0.999 

MR  0  RM  0.7  6  5  300  6  0.044  0.121  0.988  0.986 

MR  0  RM  0.7  6  15  100  3  0.005  0.572  1.346  0.971 

MR  0  RM  0.7  6  15  100  6  0.059  0.569  1.029  1.007 

MR  0  RM  0.7  6  15  300  3  0.009  0.345  1.014  1.005 

MR  0  RM  0.7  6  15  300  6  0.074  0.369  0.993  1.000 

MR  0  RM  0.7  12  5  100  3  0.001  0.210  0.946  0.990 

MR  0  RM  0.7  12  5  100  6  0.053  0.222  0.991  0.950 

MR  0  RM  0.7  12  5  300  3  0.016  0.190  1.019  0.990 

MR  0  RM  0.7  12  5  300  6  0.031  0.162  1.025  0.979 

MR  0  RM  0.7  12  15  100  3  0.015  0.882  1.306  1.014 

MR  0  RM  0.7  12  15  100  6  0.004  0.634  1.007  0.974 

MR  0  RM  0.7  12  15  300  3  0.014  0.563  1.015  0.995 

MR  0  RM  0.7  12  15  300  6  0.013  0.487  1.014  0.988 
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Table  3  0 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0,7 
0.7 


6 
6 
6 
6 
6 
6 
6 
6 
12 
12 
12 
12 


5 
5 
5 
5 
15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 


5 
5 
5 
5 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3 
6 
3 
6 
3 


0.  021 
0.007 
0.023 
0.016 
0.113 
6  0.030 
3  0.108 
6  0.061 
0.042 
0.035 
0.039 
6  0.012 
3  0.223 
6  0.123 
0.  210 
0.  076 
0.  013 
6  0.105 
3  0.029 
6  0.026 


3 
6 
3 


3 
6 
3 


0.410 
0.275 
0.412 
0.284 
0.787 
0.578 
0.781 
0.593 
0.746 
0.478 
0.746 
0.509 
1.466 
0.982 
1.490 
0.996 
0.380 
0.379 
0.443 
0.351 


2.371 
2.035 
2.386 
1.997 
2.002 
1.844 
1.991 
1.795 
2.433 
2.016 
2.347 
2.063 
2.084 
1.829 
1.985 
1.819 
2.246 
1.906 
38.38 
1.885 


0.976 
0.940 
0.990 
0.989 
1.031 
0.982 
1.036 
1.021 
0.957 

0.  955 
1.020 
1.018 
1.013 
1.008 

1.  069 
1.  060 
0.972 
0.987 
1.020 
0.978 
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Table  3  0--continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 


2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  LC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
12 
12 
12 
12 


6 
6 
6 
6 
6 
6 
6 
6 


15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 
15 
15 
15 
15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.060 
6  0.116 
3  0.083 
6  0.050 
3  0.029 
6  0.454 
3  0,037 
6  0.161 
3  0.164 
6  0.899 
3  0.163 
6  0.230 
3  0.035 
6  0.012 
0.  020 
0.010 
0.  058 
0.  036 
0.084 


6  0.024 


0.728 
0.791 
0.775 
0.741 
0.700 
0.602 
0.813 
0.599 
1.313 
1.207 
1.428 
1. 182 
0.476 
0.404 
0.481 
0.432 
0.831 
0.733 
0.831 
0.770 


1.910 
1.712 
12.13 
1.670 
2.299 
1.933 
2.371 
1.949 
1.905 
1.712 
1.913 
1.702 
8.535 
8.729 
8.919 
9.625 
4.982 
5.  162 
5.  057 
5.  350 


1.010 
1.042 
1.056 
1.012 
0.933 
0.980 
1.014 
0.979 
0.981 
1.037 
1.055 
1.023 
1.019 
1.012 

0.  998 

1.  018 
1.090 
1.099 
1.035 
1.047 
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Table  30 — continued. 


M       SK     TM  R 


PM     N        p  BSE 


RE 


MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 


2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 


0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 

0.2  12 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


6 
6 
6 
6 
6 
6 
6 
6 


0.7  12 

0.7  12 

0.7  12 

0.7  12 


5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 


6 
3 


3  0.034 
6  0.003 
3  0.012 
6  0.009 
3  0.233 
0.032 
0.144 
6  0.029 
3  0.014 
6  0.033 
0.013 
0.  028 
0.044 
6  0.073 
3  0.043 
0.  073 
0.  042 
6  0.433 
3  0.027 
6  0.017 


3 
6 
3 


6 
3 


0.751 

0.570 

0.795 

0.613 

1.399 

1.093 

1.513 

1.130 

0.528 

0.772 

0.532 

0.802 

0.816 

1.267 

0.787 

1.293 

0.780 

1.018 

0.866 

1.021 


8. 162 
8.845 
9.190 
9.521 
4.887 
5.192 
5. 104 
5.310 
8.142 
8.818 
8.735 
9.419 
4.723 
4.923 
4.852 
5.  035 
8.464 
8.938 
8.922 
9.  349 


0.980 
1.011 
1.024 
1.030 
1.061 
1.094 
1.075 
1.065 

0.  996 

1.  048 
1.  003 

0.  996 
1.055 
1.081 
1.026 

1.  023 

0.  956 
1.029 

1.  019 
0.  990 


158 


Table  30 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

MR  2  RC  0.7  12  15  100  3  0.107  1.275  4.805  1.016 

MR  2  RC  0.7  12  15  100  6  0.601  1.730  4.947  1.120 

MR  2  RC  0.7  12  15  300  3  0.102  1.328  4.912  1.047 

MR  2  RC  0.7  12  15  300  6  0.093  1.662  4.998  1.032 

MR  2  RM  0.2  6  5  100  3  0.001  0.090  1.032  0.967 

MR  2  RM  0.2  6  5  100  6  0.004  0.090  1.037  0.932 

MR  2  RM  0.2  6  5  300  3  0.002  0.103  0.957  0.979 

MR  2  RM  0.2  6  5  300  6  0.005  0.084  1.038  0.984 

MR  2  RM  0.2  6  15  100  3  0.012  0.268  0.964  0.981 

MR  2  RM  0.2  6  15  100  6  0.005  0.238  1.070  0.967 

MR  2  RM  0.2  6  15  300  3  0.014  0.297  0.981  0.987 

MR  2  RM  0.2  6  15  300  6  0.011  0.235  0.996  0.994 

MR  2  RM  0.2  12  5  100  3  0.002  0.167  0.896  0.944 

MR  2  RM  0.2  12  5  100  6  0.008  0.134  0.964  0.945 

MR  2  RM  0.2  12  5  300  3  0.002  0.183  1.042  1.010 

MR  2  RM  0.2  12  5  300  6  0.001  0.127  0.982  1.008 

MR  2  RM  0.2  12  15  100  3  0.014  0.498  1.000  0.956 

MR  2  RM  0.2  12  15  100  6  0.011  0.373  1.009  0.968 

MR  2  RM  0.2  12  15  300  3  0.014  0.527  0.995  1.015 

MR  2  RM  0.2  12  15  300  6  0.007  0.374  0.982  1.016 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
MR 
PP 
PP 
PP 
PP 


2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

0  LC 

0  LC 

0  LC 

0  LC 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
6 
6 
6 
6 
12 
12 
12 
12 


5 
5 
5 
5 
15 
15 
15 
15 
5 
5 
5 
5 


100 
100 


3 
6 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 


5 
5 
5 
5 


300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 

100  3 

100  6 

300  3 

300  6 


0.  001 
0.047 
0.  034 
0.  002 
0.  005 
0.  029 
0.026 
0.  005 
0.019 
0.082 
0.  006 
0.  030 
0.  007 
0.474 
0.  012 
0.  085 
0.  016 
0.  005 
0.023 
0.011 


0.106 
0.121 
0.111 
0. 122 
0.331 
0.350 
0.340 
0.341 
0.176 
0. 166 
0.210 
0. 179 
0.554 
0.498 
0.550 
0.504 
0.464 
0.372 
0.346 
0.276 


1.032 
1.020 

0.  986 
1.032 

1.  047 
1.018 
1.030 
0.999 
0.999 

0.  989 
1.042 
1.013 
1.002 

1.  020 
1.000 
1.030 
0.  000 
0.  000 
0.000 
0.000 


0.963 
0.980 
1.012 
0.975 
0.981 
1.016 
1.020 
0.991 
0.922 
0.970 
1.003 
0.971 
0.936 
1.001 
1.007 
0.982 
1.012 
0.  984 
0.988 
1.023 
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Table  30 — continued. 


M 


SK     TM  R 


PM     N         p  BSE 


D 


RE 


PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 


0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


6 
6 
6 
6 
12 
12 
12 
12 


6 
6 
6 
6 
6 
6 
6 
6 


15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 
15 
15 
15 
15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3 
6 
3 


3 
6 
3 


0.073 
0.  015 
0.076 
6  0.022 
3  0.024 
6  0.006 
0.037 
0.005 
0. 151 
6  0.014 
3  0.151 
6  0.024 
0.018 
0.214 
0.011 
0.021 
0.  051 
0.256 
0.  050 


6  0.047 


0.810 
0.672 
0.612 
0.518 
0.667 
0.518 
0.657 
0.450 
1.371 
0.934 
1.160 
0.828 
0.510 
0.590 
0.370 
0.434 
0.714 
0.913 
0.591 
0.823 


0.  000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.000 
0.  000 
0.  000 
0.000 
0.  000 
0.000 
0.  000 
0.  000 
0.  000 
0.000 
0.  000 
0.  000 
0.  000 


1.090 
1.058 
1.020 
1.054 
1.019 
0.977 
1.010 
1.002 
1.096 
1.044 
1.047 
1.031 
0.972 
0.987 
1.005 

0.  992 

1.  034 
1.  042 
1.030 
1.020 
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Table  3  0 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


RE 


PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 


0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 


0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
6 
6 
6 
6 


0.2  12 

0.2  12 

0.2  12 

0.2  12 


5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 


3 
6 
3 
6 
3 


3  0.016 
6  0.023 
0.022 
0.103 
0. 114 
0.035 
0.095 
6  0.091 
3  0.014 
6  0.006 
0.019 
0.011 
0.086 
6  0.018 
3  0.072 
0.025 
0.015 
0.009 
0.020 


3 
6 
3 


5     300     6  0.015 


0.719 
0.650 
0.656 
0.559 
1.220 
1.129 
1.014 
1.080 
0.422 
0.393 
0.353 
0.288 
0.818 
0.712 
0.618 
0.540 
0.  628 
0.515 
0.634 
0.465 


0.  000 
0.  000 
0.  000 
0,000 
0.  000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.  000 
0.000 
0.  000 
0.000 
0.  000 
0.  000 
0.  000 
0.  000 
0.000 
0  .  000 


1.010 

0.970 
0.999 

0.  987 
1.078 
1.014 
1.032 
1.016 

1.  Oil 
0.985 

0.  989 
1.026 
1. 100 

1.  067 
1.020 
1.054 
1.013 

0.  978 
1.007 

1.  004 
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Table  3  0 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 


0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 


0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 
6 
6 
6 
6 
12 
12 
12 
12 


5 
5 
5 
5 
15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3 
6 
3 


3  0.148 
6  0.036 
0.116 
0.060 
0.005 
6  0.213 
3  0.012 
6  0.083 
3  0.019 
6  0.067 
3  0.048 
6  0.095 
0.016 
0.010 
0.052 
6  0.043 
3  0.133 
6  0.018 
3  0.158 
6  0.090 


3 
6 
3 


1.339 
0.948 
1.153 
0.859 
0.449 
0.548 
0.378 
0.422 
0.727 
0.895 
0.578 
0.809 
0.718 
0.661 
0.  681 

0.  574 
1. 134 

1.  082 
1.070 
1.084 


0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.000 
0.000 


1.  090 
1.033 
1.044 
1.040 
0.967 
0.988 
1.008 

0.  993 
1.009 

1.  043 
1.034 
1.020 
1.004 
0.972 
1.002 

0.  987 

1.  067 
1.024 
1.032 
1.  016 
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Table  3  0 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


RE 


PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 


0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 


6 
6 
6 
6 


12 
12 
12 
12 


6 
6 
6 
6 


5 
5 
5 
5 


6  15 

6  15 

6  15 

6  15 


5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.000 

6  0.001 

3  0.002 

6  0.002 

3  0.002 

6  0.001 

3  0.004 

6  0.004 

3  0.000 

6  0.001 

3  0.001 

6  0.003 

3  0.002 
0.004 
0.000 

6  0.005 

3  0.001 

6  0.121 

3  0.001 


6 
3 


6  0.068 


0.  085 
0.065 
0.037 
0.032 
0.153 
0.119 
0.068 
0.056 
0.095 
0.085 
0.059 
0.048 
0.268 
0. 168 
0. 109 
0.  086 
0.087 
0.330 
0.042 
0.  036 


0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.  000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0,  000 
0.000 
0.  000 


0.991 
0.949 
0.978 
1.013 
1.014 
0.990 
0.982 
1.024 
1.000 
0.950 
0.997 
0.991 
1.019 

0.  982 

1.  001 
1.001 

0.  954 

1.  017 
0.999 
0.  990 


Table  30 — continued. 
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M       SK     TM  R 


PM     N         p  BSE 


RE 


PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 


0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 


2 
2 
2 
2 
2 
2 
2 
2 


LC 
LC 
LC 
LC 
LC 
LC 
LC 
LC 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6  15 

6  15 

6  15 

6  15 


12 
12 
12 
12 


6 
6 
6 
6 
6 
6 
6 
6 


5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 
15 
15 
15 
15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.769 
6  0.187 
3  0.002 
6  0.088 
3  0.001 
6  0.598 
3  0.017 
6  0.142 
3  0.039 
0.267 
0.006 
0.308 
0.015 
6  0.005 
3  0.017 
0.  013 
0.080 
6  0.017 
3  0.075 
6  0.041 


6 
3 
6 
3 


6 
3 


0.176 
0.538 
0.083 
0.144 
0.133 
0.110 
0.054 
0.621 
0.314 
0.287 
0.099 
1.021 
0.333 
0.214 
0.333 
0.222 
0.  603 
0.425 
0.595 
0.438 


0.000 
0.000 
0.  000 
0.  000 
0.  000 
0.000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.000 
0.  000 
0.000 
0.  000 
0.000 
0.  000 
0.000 
0.  000 


1.401 
1.123 
1.011 
1.  016 
0.991 
1.022 
0.990 
0.987 
1.064 
1.219 

0.  998 

1.  001 
0.975 
0.939 
0.989 

0.  987 

1.  023 

0.  978 

1.  026 
1.010 
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Table  30 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


RE 


PP  2  LC  0.2  12 

PP  2  LC  0.2  12 

PP  2  LC  0.2  12 

PP  2  LC  0.2  12 

PP  2  LC  0.2  12 

PP  2  LC  0.2  12 

PP  2  LC  0.2  12 

PP  2  LC  0.2  12 

PP  2  LC  0.7  6 

PP  2  LC  0.7  6 

PP  2  LC  0.7  6 

PP  2  LC  0.7  6 

PP  2  LC  0.7  6 

PP  2  LC  0.7  6 

PP  2  LC  0.7  6 

PP  2  LC  0.7  6 

PP  2  LC  0.7  12 

PP  2  LC  0.7  12 

PP  2  LC  0.7  12 

PP  2  LC  0.7  12 


5  100  3  0.035  0.611  0.000  0.957 

5  100  6  0.029  0.382  0.000  0.955 

5  300  3  0.032  0.610  0.000  1.019 

5  300  6  0.010  0.408  0.000  1.017 

15  100  3  0.181  1.150  0.000  1.009 

15  100  6  0.092  0.751  0.000  1.008 

15  300  3  0.168  1.154  0.000  1.062 

15  300  6  0.055  0.759  0.000  1.051 

5  100  3  0.011  0.337  0.000  0.970 

5  100  6  0.169  0.277  0.000  1.002 

5  300  3  0.027  0.381  0.000  1.021 

5  300  6  0.024  0.251  0.000  0.976 

15  100  3  0.041  0.502  0.000  1.006 

15  100  6  0.264  0.511  0.000  1.034 

15  300  3  0.062  0.563  0.000  1.052 

15  300  6  0.033  0.485  0.000  1.007 

5  100  3  0.025  0.559  0.000  0.932 

5  100  6  0.105  0.453  0.000  0.980 

5  300  3  0.032  0.660  0.000  1.013 

5  300  6  0.134  0.456  0.000  0.977 
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Table  30 — continued. 


M 

SK 

TM 

R 

E 

PM 

N 

P 

BSE 

D 

Q 

RE 

PP 

2 

LC 

0. 

7 

12 

15 

100 

3 

0. 132 

0. 

944 

0. 

000 

0.  978 

PP 

2 

LC 

0. 

7 

12 

15 

100 

6 

0. 129 

0. 

850 

0. 

000 

1. 118 

PP 

2 

LC 

0. 

7 

12 

15 

300 

3 

0. 135 

1. 

067 

0 . 

000 

1.  049 

PP 

2 

LC 

0. 

7 

12 

15 

300 

6 

0.  266 

0 . 

847 

0 . 

000 

1.015 

PP 

2 

RC 

0. 

2 

6 

5 

100 

3 

0.  030 

0. 

407 

0. 

000 

1.  021 

PP 

2 

RC 

0. 

2 

6 

5 

100 

6 

0.  012 

0. 

353 

0. 

000 

1 . 019 

PP 

2 

RC 

0. 

2 

6 

5 

300 

3 

0.014 

0. 

409 

0. 

000 

0.987 

PP 

2 

RC 

0. 

2 

6 

5 

300 

6 

0.009 

0. 

381 

0. 

000 

1.019 

PP 

2 

RC 

0. 

2 

6 

15 

100 

3 

0.  037 

0 . 

666 

0. 

000 

1. 110 

PP 

2 

RC 

0 . 

2 

6 

15 

100 

6 

0 .  032 

0. 

616 

0 . 

000 

1. 141 

PP 

2 

RC 

0 . 

2 

6 

15 

300 

3 

0 .  056 

0 . 

663 

0 . 

000 

1 .  031 

PP 

2 

RC 

0 . 

2 

6 

15 

300 

6 

0 ,  017 

0 . 

654 

0 . 

000 

1 .  055 

PP 

2 

RC 

0 . 

2 

12 

5 

100 

3 

0.028 

0 . 

619 

0 . 

000 

0.  982 

PP 

2 

RC 

0 . 

2 

12 

5 

100 

6 

0 .  002 

0. 

482 

0. 

000 

1.  019 

PP 

2 

RC 

0 . 

2 

12 

5 

300 

3 

0.  008 

0. 

665 

0. 

000 

1.  024 

PP 

2 

RC 

0. 

2 

12 

5 

300 

6 

0.008 

0. 

520 

0. 

000 

1.  031 

PP 

2 

RC 

0. 

2 

12 

15 

100 

3 

0.206 

1. 

084 

0. 

000 

1.  074 

PP 

2 

RC 

0. 

2 

12 

15 

100 

6 

0.025 

0. 

887 

0. 

000 

1. 131 

PP 

2 

RC 

0. 

2 

12 

15 

300 

3 

0. 108 

1. 

189 

0. 

000 

1.  076 

PP 

2 

RC 

0. 

2 

12 

15 

300 

6 

0.015 

0. 

921 

0. 

000 

1.  073 
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Table  30 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


RE 


PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 
PP 


2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RC 

2  RM 

2  RM 

2  RM 

2  RM 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
6 
6 
6 
6 
12 
12 
12 
12 


5 
5 
5 
5 
15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 


5 
5 
5 
5 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.013 

6  0.034 

3  0.010 

6  0.030 

3  0.043 

6  0.082 

3  0.034 

6  0.076 

3  0.038 

6  0.419 

3  0.023 

6  0.014 

3  0.087 

6  0.700 

3  0.084 

6  0.094 

3  0.001 

6  0.004 

3  0.000 

6  0.004 


0.456 
0.698 
0.459 
0.731 
0.646 
1.113 
0.607 
1. 148 
0.656 
0.901 
0.726 
0.908 
0.943 
1.485 
0.990 
1.426 
0.060 
0.  058 
0.034 
0.  034 


0.  000 
0.  000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.000 
0.000 
0.000 
0.  000 
0.000 
0.  000 
0.000 
0.  000 
0.000 
0.  000 
0.  000 
0.  000 
0.  000 


0.996 
1.052 
1.002 
2.989 
1.071 
1. 105 
1.026 
3.078 
0.958 
1.034 
1.018 
0.990 
1.031 
1.150 
1.048 
1.032 
0.968 
0.932 
0.979 
0.984 
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Table  3  0 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


PP  2  RM  0.2  6  15  100  3  0.003  0.116  0.000  0.983 

PP  2  RM  0.2  6  15  100  6  0.003  0.114  0.000  0.986 

PP  2  RM  0.2  6  15  300  3  0.001  0.063  0.000  0.984 

PP  2  RM  0.2  6  15  300  6  0.005  0.063  0.000  0.996 

PP  2  RM  0.2  12  5  100  3  0.001  0.105  0.000  0.944 

PP  2  RM  0.2  12  5  100  6  0.008  0.080  0.000  0.947 

PP  2  RM  0.2  12  5  300  3  0.001  0.062  0.000  1.010 

PP  2  RM  0.2  12  5  300  6  0.000  0.037  0.000  1.008 

PP  2  RM  0.2  12  15  100  3  0.005  0.200  0.000  0.959 

PP  2  RM  0.2  12  15  100  6  0.005  0.150  0.000  0.981 

PP  2  RM  0.2  12  15  300  3  0.002  0.108  0.000  1.015 

PP  2  RM  0.2  12  15  300  6  0.001  0.086  0.000  1.019 

PP  2  RM  0.7  6  5  100  3  0.001  0.066  0.000  0.965 

PP  2  RM  0.7  6  5  100  6  1.719  0.074  0.000  1.101 

PP  2  RM  0.7  6  5  300  3  0.037  0.039  0.000  1.012 

PP  2  RM  0.7  6  5  300  6  3.152  0.050  0.000  1.008 

PP  2  RM  0.7  6  15  100  3  0.058  0.143  0.000  1.043 

PP  2  RM  0.7  6  15  100  6  1.024  0.142  0.000  1.508 

PP  2  RM  0.7  6  15  300  3  0.038  0.078  0.000  1.028 

PP  2  RM  0.7  6  15  300  6  1.638  0.118  0.000  1.414 
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Table  3  0 — continued. 


M       SK     TM  R 


PM     N        p  BSE 


RE 


PP  2  RM  0.7  12 

PP  2  RM  0.7  12 

PP  2  RM  0.7  12 

PP  2  RM  0.7  12 

PP  2  RM  0.7  12 

PP  2  RM  0.7  12 

PP  2  RM  0.7  12 

PP  2  RM  0.7  12 

RG  0  LC  0.2 

RG  0  LC  0.2 

RG  0  LC  0.2 


RG  0  LC  0.2 
RG       0     LC  0.2 


RG 
RG 
RG 
RG 


0  LC  0.2 
0  LC  0.2 
0     LC  0.2 


6 
6 
6 
6 
6 
6 
6 
6 


0     LC     0.2  12 


RG  0  LC  0.2  12 
RG  0  LC  0.2  12 
RG       0     LC     0.2  12 


5  100  3  0.042  0.096  0.000  0.924 

5  100  6  1.812  0.094  0.000  1.036 

5  300  3  0.009  0.071  0.000  1.004 

5  300  6  1.489  0.064  0.000  1.019 

15  100  3  0.782  0.202  0.000  1.342 

15  100  6  0.540  0.226  0.000  1.365 

15  300  3  0.014  0.116  0.000  1.012 

15  300  6  2.698  0.122  0.000  1.025 

5  100  3  0.003  0.504  3.826  1.001 

5  100  6  0.004  0.396  3.469  0.964 

5  300  3  0.006  0.389  4.007  0.982 

5  300  6  0.006  0.306  3.579  1.019 

15  100  3  0.013  0.881  3.449  1.042 

15  100  6  0.028  0.733  3.103  1.033 

15  300  3  0.007  0.680  2.965  0.993 

15  300  6  0.016  0.578  3.112  1.050 

5  100  3  0.006  0.759  3.977  1.013 

5  100  6  0.005  0.589  3.586  0.972 

5  300  3  0.010  0.765  4.238  1.005 

5  300  6  0.002  0.524  3.765  1.000 


Table  30 — continued. 
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M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 


0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 

0  LC 


0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 
6 
6 


12 
12 
12 
12 


5 
5 
5 
5 
15 
15 


6  15 
6  15 


5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.005 

6  0.021 

3  0.021 

6  0.007 

3  0.016 

6  0.349 

3  0.013 

6  0.229 

3  0.008 

6  3.893 

3  0.021 

6  0.446 

3  0.018 
0.347 
0.  015 

6  0.051 

3  0.046 

6  1.715 

3  0.027 


6 
3 


6  0.136 


1.604 

1.  080 

1.352 

0.963 

0.514 

0.663 

0.400 

0.462 

0.712 

1.113 

0.653 

0.881 

0.800 

0.743 

0.759 

0.624 

1.456 

1.333 

1.236 

1.221 


3  .858 
3.355 
3.091 
3.141 
1.962 
3.798 
2.107 
1.060 
2.169 
4.490 
1.831 

0.  998 
2.166 
3.321 
2.533 

1.  109 
2.692 
3.353 
2  .  048 
1. 120 


1.059 
1.046 
1.021 
1.020 
0.965 
1.152 
1.003 
1.021 

0.  982 
1.530 
1.010 

1.  054 
1.001 
1.047 

0.  998 
1.001 
1.038 
1.208 

1.  Oil 
1.  037 
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Table  30 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

RG  0  RC  0.2  6  5  100  3  0.030  0.499  3.309  1.019 

RG  0  RC  0.2  6  5  100  6  0.037  0.441  2.820  1.006 

RG  0  RC  0.2  6  5  300  3  0.022  0.415  3.474  0.994 

RG  0  RC  0.2  6  5  300  6  0.031  0.332  2.818  1.037 

RG  0  RC  0.2  6  15  100  3  0.131  1.051  3.021  1.129 

RG  0  RC  0.2  6  15  100  6  0.098  0.850  2.515  1.128 

RG  0  RC  0.2  6  15  300  3  0.066  0.773  2.542  1.034 

RG  0  RC  0.2  6  15  300  6  0.074  0.657  2.387  1.091 

RG  0  RC  0.2  12  5  100  3  0.021  0.744  3.475  1.017 

RG  0  RC  0.2  12  5  100  6  0.034  0.605  3.189  0.988 

RG  0  RC  0.2  12  5  300  3  0.026  0.767  3.739  1.011 

RG  0  RC  0.2  12  5  300  6  0.032  0.554  3.123  1.012 

RG  0  RC  0.2  12  15  100  3  0.139  1.742  3.204  1.105 

RG  0  RC  0.2  12  15  100  6  0.118  1.171  2.852  1.101 

RG  0  RC  0.2  12  15  300  3  0.099  1.469  2.756  1.052 

RG  0  RC  0.2  12  15  300  6  0.081  1.068  2.621  1.061 

RG  0  RC  0.7  6  5  100  3  0.055  0.507  2.559  1.001 

RG  0  RC  0.7  6  5  100  6  0.721  0.672  6.513  1.352 

RG  0  RC  0.7  6  5  300  3  0.045  0.430  2.845  1.030 

RG  0  RC  0.7  6  5  300  6  0.769  0.479  1.578  1.085 


Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 


0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RC 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 

0  RM 


0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6  15 

6  15 

6  15 

6  15 


12 
12 
12 
12 


6 
6 
6 
6 
6 
6 
6 
6 


5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


5 
5 
5 
5 
15 
15 
15 
15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


6 
3 


3  0.249 
6  3.473 
3  0.129 
6  1.556 
3  0.071 
1.144 
0.089 
6  0.076 
3  0.387 
1.216 
0.246 
6  0.411 
3  0.002 
0.  Oil 
0.003 
0.008 
0.  043 
0.  091 
0.  019 


6 
3 


6  0.050 


0.917 
1.317 
0.727 
0.976 
0.832 
0.774 
0.809 
0.661 
1.613 
1.457 
1.391 
1.328 
0.145 
0. 136 
0.087 
0.  084 
0.451 
0.408 
0.247 
0.239 


2.670 
6.339 
2.208 
1.141 
2.863 
4.704 
2.697 
1.335 
2.862 
5.003 
2.077 
1.122 
1.833 
2.518 
1.545 
2.046 
2.372 
3  .509 
1.586 
2.228 


1. 141 
1.978 
1.095 
1.304 
1.026 
1. 131 
1.010 
1.030 
1. 145 
1.463 
1.053 
1. 165 
0.994 
0.965 

0.  980 

1.  018 
1.  046 
1.  108 

0.  997 

1.  069 
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Table  3  0 — continued. 


M       SK     TM  R 


PM    N        p  BSE 


RE 


RG  0  RM  0.2  12 
RG       0     RM     0.2  12 


0  RM  0.2  12 
0     RM     0.2  12 


RG  0  RM  0.2  12 
RG 
RG 

RG  0  RM  0.2  12 

RG  0  RM  0.2  12 

RG  0  RM  0.2  12 

RG  0  RM  0.7  6 

RG  0  RM  0.7  6 

RG  0  RM  0.7  6 

RG  0  RM  0.7  6 

RG  0  RM  0.7  6 

RG  0  RM  0.7  6 

RG  0  RM  0.7  6 

RG  0  RM  0.7  6 
RG 
RG 
RG 


0  RM  0.7  12 
0     RM     0.7  12 


RG 


0  RM  0.7  12 
0     RM     0.7  12 


5  100  3  0.003  0.175  1.578  1.002 

5  100  6  0.006  0.177  2.238  0.961 

5  300  3  0.003  0.173  1.493  1.000 

5  300  6  0.011  0.130  1.804  0.994 

15  100  3  0.035  0.779  2.442  1.056 


15  100 


0.055     0.491     2.750  1.045 


15  300  3  0.034  0.501  1.643  1.016 

15  300  6  0.029  0.362  2.023  1.026 

5  100  3  0.012  0.125  0.806  0.955 

5  100  6  0.050  0.153  0.557  0.985 

5  300  3  0.007  0.094  0.780  1.000 

5  300  6  0.816  0.093  0.318  1.003 

15  100  3  0.165  0.462  1.433  1.040 

15  100  6  2.930  0.471  1.310  1.416 

15  300  3  0.080  0.283  0.876  1.028 

15  300  6  1.521  0.293  0.591  1.233 

5  100  3  0.005  0.191  0.825  0.992 

5  100  6  0.249  0.177  0.364  0.974 

5  300  3  0.015  0.171  0.733  0.992 

5  300  6  0.041  0.139  0.296  0.988 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D    ^         Q  RE 


RG  0  RM  0.7  12  15  100  3  0.142  0.792  1.470  1.074 

RG  0  RM  0.7  12  15  100  6  0.125  0.541  0.846  1.210 

RG  0  RM  0.7  12  15  300  3  0.039  0.518  0.867  1.019 

RG  0  RM  0.7  12  15  300  6  0.423  0.427  0.516  1.170 

RG  2  LC  0.2  6  5  100  3  0.005  0.379  2.544  0.968 

RG  2  LC  0.2  6  5  100  6  0.004  0.250  2.397  0.937 

RG  2  LC  0.2  6  5  300  3  0.004  0.383  2.443  0.982 

RG  2  LC  0.2  6  5  300  6  0.012  0.261  2.077  0.985 

RG  2  LC  0.2  6  15  100  3  0.010  0.674  2.247  0.982 

RG  2  LC  0.2  6  15  100  6  0.006  0.494  2.678  0.966 

RG  2  LC  0.2  6  15  300  3  0.003  0.672  2.154  0.986 

RG  2  LC  0.2  6  15  300  6  0.026  0.512  2.336  0.994 

RG  2  LC  0.2  12  5  100  3  0.012  0.713  2.814  0.952 

RG  2  LC  0.2  12  5  100  6  0.016  0.457  2.344  0.951 

RG  2  LC  0.2  12  5  300  3  0.012  0.719  2.543  1.015 

RG  2  LC  0.2  12  5  300  6  0.006  0.486  2.059  1.014 

RG  2  LC  0.2  12  15  100  3  0.028  1.323  2.508  0.969 

RG  2  LC  0.2  12  15  100  6  0.012  0.892  2.619  0.982 

RG  2  LC  0.2  12  15  300  3  0.030  1.356  2.257  1.025 

RG  2  LC  0.2  12  15  300  6  0.018  0.902  2.195  1.022 
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Table  3  0 — continued. 


M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

RG  2  LC  0.7  6  5  100  3  0.012  0.344  1.665  0.966 

RG  2  LC  0.7  6  5  100  6  0.117  0.340  0.802  0.997 

RG  2  LC  0.7  6  5  300  3  0.021  0.423  39.80  1.018 

RG  2  LC  0.7  6  5  300  6  0.215  0.315  0.776  0.959 

RG  2  LC  0.7  6  15  100  3  0.020  0.607  2.036  0.981 

RG  2  LC  0.7  6  15  100  6  0.211  0.668  0.796  1.044 

RG  2  LC  0.7  6  15  300  3  0.032  0.673  13.48  1.026 

RG  2  LC  0.7  6  15  300  6  0.074  0.625  0.762  1.020 

RG  2  LC  0.7  12  5  100  3  0.024  0.669  2.460  0.927 

RG  2  LC  0.7  12  5  100  6  0.037  0.563  0.975  0.975 

RG  2  LC  0.7  12  5  300  3  0.025  0.781  2.187  1.010 

RG  2  LC  0.7  12  5  300  6  0.034  0.564  0,874  0.983 

RG  2  LC  0.7  12  15  100  3  0.075  1.193  2.646  0.953 

RG  2  LC  0.7  12  15  100  6  0.550  1.067  0.961  1.020 

RG  2  LC  0.7  12  15  300  3  0.063  1.303  2.236  1.026 

RG  2  LC  0.7  12  15  300  6  0.071  1.053  0.878  1.010 

RG  2  RC  0.2  6  5  100  3  0.031  0.455  6.529  1.024 

RG  2  RC  0.2  6  5  100  6  0.031  0.382  5.886  1.031 

RG  2  RC  0.2  6  5  300  3  0.013  0.462  6.566  1.001 

RG  2  RC  0.2  6  5  300  6  0.012  0.413  6.202  1.017 


lie 

Table  30 — continued. 

M  SK  TM  R  E  PM  N  p  BSE  D  Q  RE 

RG  2  RC  0.2  6  15  100  3  0.051  0.807  3.964  1.147 

RG  2  RC  0.2  6  15  100  6  0.165  0.718  4.030  1.303 

RG  2  RC  0.2  6  15  300  3  0.036  0.809  3.787  1.045 

RG  2  RC  0.2  6  15  300  6  0.040  0.755  3.947  1.112 

RG  2  RC  0.2  12  5  100  3  0.034  0.737  6.474  0.996 

RG  2  RC  0.2  12  5  100  6  0.019  0.553  6.438  1.029 

RG  2  RC  0.2  12  5  300  3  0.019  0.781  7.066  1.030 

RG  2  RC  0.2  12  5  300  6  0.011  0.600  6.809  1.039 

RG  2  RC  0.2  12  15  100  3  0.200  1.385  4.028  1.134 

RG  2  RC  0.2  12  15  100  6  0.139  1.077  4.273  1.220 

RG  2  RC  0.2  12  15  300  3  0.086  1.501  3.995  1.117 

RG  2  RC  0.2  12  15  300  6  0.046  1.117  4.133  1.154 

RG  2  RC  0.7  6  5  100  3  0.063  0.453  3.848  1.020 

RG  2  RC  0.7  6  5  100  6  0.213  0.720  3.048  1.248 

RG  2  RC  0.7  6  5  300  3  0.043  0.447  3.705  1.016 

RG  2  RC  0.7  6  5  300  6  0.176  0.759  3.319  1.149 

RG  2  RC  0.7  6  15  100  3  0.185  0.719  2.563  1.177 

RG  2  RC  0.7  6  15  100  6  0.890  1.220  2.075  1.760 

RG  2  RC  0.7  6  15  300  3  0.069  0.671  2.245  1.071 

RG  2  RC  0.7  6  15  300  6  1.404  1.253  2.061  1.511 
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Table  30 — continued. 


M       SK     TM  R 


PM     N         p  BSE 


RE 


RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 


2 
2 
2 
2 
2 
2 
2 
2 
2 
2 


2 
2 
2 


2 
2 


RC 
RC 
RC 
RC 
RC 
RC 
RC 
RC 
RM 
RM 


2  RM 
2  RM 


RM 
RM 
RM 


2  RM 


RM 
RM 


0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 

0.7  12 


0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 
0.2 


6 
6 
6 
6 
6 
6 
6 
6 


2  RM 


2  RM 


0.2  12 

0.2  12 

0.2  12 

0.2  12 


5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 

15  100 

15  100 

15  300 

15  300 

5  100 

5  100 

5  300 

5  300 


3  0.089 
6  0.248 
3  0.040 
6  0.191 
3  0.204 
6  0.924 
0. 116 
1.297 
0.004 
0.  009 
0.002 
6  0.009 
3  0.022 
6  0.038 
3  0.018 
6  0.093 
3  0.008 
0.008 
0.  007 
0.  003 


3 
6 
3 
6 
3 


0.728 
0.974 
0.819 
0.987 
1.216 
1.687 
1.271 
1.624 
0.  093 
0.097 
0.094 
0.  084 
0.252 
0.273 
0.265 
0.239 
0. 175 
0. 149 
0. 178 
0.  125 


4.252 
3.128 
4.103 
3.326 
3.065 
2.129 
2.675 
2.101 
1.904 
2.335 
1.563 

2  . 169 
1.843 

3  .  079 
1.600 
2.384 
1.700 
2.152 
1.746 
1.725 


0.972 
1.138 
1.027 
1.090 
1.116 
1.438 
1.115 
1.311 
0.972 
0.943 
0.980 
0.990 
1.003 
1.080 
0.997 
1.042 
0.948 

0.  955 
1.012 

1.  012 
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Table  30 — continued. 


M       SK     TM     R         E       PM     N         p     BSE         D  Q  RE 


RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 
RG 


2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 

2  RM 


0.2 
0.2 
0.2 
0.2 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0.7 
0,7 
0.7 
0.7 


12  15 

12  15 

12  15 

12  15 


6 
6 
6 
6 
6 
6 
6 
6 
12 
12 
12 
12 


5 
5 
5 
5 
15 
15 
15 
15 
5 
5 
5 
5 


12  15 

12  15 

12  15 

12  15 


100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 
100 
100 
300 
300 


3  0.053 
6  0.087 
3  0.036 
6  0.026 
3  0.004 
6  0.820 
0.007 
0. 153 
0.  098 
1.061 
0.  028 
6  2.386 
3  0.009 
6  0.056 
0.  002 
0.541 
0.  054 
1.430 
0.  094 
2  .436 


3 
6 
3 
6 
3 


0.511 
0.420 
0.507 
0.375 
0.087 
0.093 
0.090 
0.092 
0.279 
0.291 
0.278 
0.268 
0. 163 
0. 141 
0. 191 
0.155 
0.514 
0.448 
0.505 
0.442 


2.255 
3.042 
1.693 
2  .  038 
0.768 
0.416 
0.739 
0.350 
1.068 
0.896 
0.823 
0.613 
0.891 
0.465 
0.750 
0.343 
1.015 
0.912 
0.  860 
0.  545 


0.989 
1.060 
1.028 
1.053 
0.966 
0.993 
1.013 
0.984 
1.022 
1.287 
1.036 
1.155 
0.925 

0.  982 

1.  004 
0.985 

0.  975 
1.263 

1.  027 
1. 147 
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